On Wed, Nov 29, 2017 at 10:06:40AM +0000, Tristan Van Berkom wrote:
Hi,
I'd like to start a thread on our mailing list to address this concern
which was brought up on d-d-l:
https://mail.gnome.org/archives/desktop-devel-list/2017-November/msg00070.html
In summary, some people like to use IDE features which give them symbol
browsers and symbol completion features based on some introspection and
context of what the user is editing.
Hi, I've also been looking at how to incorporate a BuildStream based workflow into an IDE and specifically VSCode which is ported to Linux, Windows and macOS. Rather than the browsing and completion side of things, I have mainly been focusing on proving an acceptable debugging and edit/compile/test cycle, which are other important aspects of the IDE experience.
For this purpose, the user needs to have a sysroot of sorts, a filesystem tree where header files, girs and basically the developer metadata installed by the modules which the project depends on. Using BuildStream, there is no single filesystem tree which makes sense, one build might have a slightly different tree; depending on what exact version (usually git commit sha) of each dependency is active for a given build, however; whipping up a build directory is something that BuildStream obviously does, it's a pretty core feature.
As you indicate there is no single 'sysroot', even from a logical point of view so I wonder if focusing on trying to create one is not the best approach, but below, it sounds like you are considering creating multiple sysroots if I understand correctly.
So, to help users who want to use these kind of IDE features I'd like
to brainstorm a bit about what the feature should look like.
It should:
a.) Be easy to use for a regular user, the user should be able to
obtain a sysroot they can feed to their IDE of choice.
b.) Be interoperable with IDEs like GNOME Builder, such that it is
possible for an IDE to automatically introspect where an
appropriate sysroot can be found for a given element.
After a week of this brewing in the background, I feel like the most
elegant solution might be to extend workspaces a bit to work with this.
In my current "proof of concept" environment I use workspaces to achieve a reasonable debugging workflow. (I have some extra hoops to jump through because my IDE is running on macOS and the code which I'm debugging is targeted at Linux so I am running BuildStream and the application under test in a container environment.) I've been working with a version of BuildStream with https://gitlab.com/BuildStream/buildstream/merge_requests/126 merged in as incremental builds are crucial to a fast compile cycle. My basic approach has been to open a workspace for a build element that builds, say, a unit test and to set up the debugger (gdb) to launch from a `bst shell --build`. VSCode needs to be told the map between the path inside the shell environment and where the workspace is outside the shell but once this is done you can step through code using the VSCode interface and perform other actions like setting breakpoints and running to a specific source line. It's possible to open a workspace for other elements to allow stepping into library code but at this point it becomes pretty important to have a different build-root for each build element. At the moment I am overriding this manually in the projects that I am interested in, but I think it would be nice to have the default build-root for a build element to be a subdirectory of /buildstream/build base on the relative path of the corresponding .bst from the project's element-path. This means that you can have a single source file mapping for the debugger from /buildstream/build to a workspaces parent folder providing you open workspaces at the same consistent relative subdirectory.
Creating checkouts on demand, even if we add a `--hardlink` option to the checkout command for optimization, when it must be done explicitly, periodically, this is unwieldy. This means that every time the latest versions of things change; the user is required to update their checkout themselves in order to have the correct development headers to develop against.
I have found checkouts useful for debugging for one reason. If you have code in a header file of a dependency then the path embedded in the debug information is the installed path and not its original path in the source element. Well actually it's worse: if you have an inline function that's used both in the build element where it is defined and also by its dependencies which use it from its installed path then it may appear in multiple locations in the same executable. Sometimes the paths to installed header files don't match how they appear in the source (frequently directory trees are flattened), and sometimes header files are generated so don't even appear explicitly in the source. Mostly an open workspace is more useful than a checkout as it has source files in addition to header files.
From a workspace point of view however, there is an opportunity to update a fixed sysroot every time we do tracking, fetching or building operations (every time that the weak cache keys of the dependencies have changed and buildstream is run, or something along those lines). These workspace sysroots would be hardlinked staging grounds, and the user MUST not ever open those files for writing, lest they go and corrupt their local artifact caches. As they would belong to the workspace, opening and closing the workspace would automatically create or delete the said sysroot.
I don't think I have a good understanding of your vision of workspace sysroots. Are you envisioning these sysroots to be `bst shell` (i.e. not `--build`) like roots with the complete execution environment available for an element, but not its source? How would you envision being able to debug some library code invoked by a task built in a dependent build element?
The command `bst workspace list` which outputs machine readable YAML
would also have to output the given corresponding sysroot for an active
workspace.
There are some pretty serious edge cases to consider further here
though, which also make me hesitant about this plan, which includes
some additional local state management which is undesirable as well:
o what happens when `bst build --track` is run, and the newly tracked
source references are never saved anymore ?
Is updating the local state at `bst build --track` time enough ?
Would we have to shadow the local state of actual checked out
versions in the local buildstream state directory in order
to keep track of what needs updating ?
o When the user updates their project via git (like pulling the
latest of the GNOME modulesets over git), buildstream is not
informed of this, so maybe this alone is a good argument to
have a less controlled approach, and demand that something
external create checkouts on demand ?
o Workspaces are Source based, an element can have more than
one source; is the structure correct to have a sysroot tied
to a given source of an element ?
What do people think about this approach ? Does anyone have other ideas
of how to elegantly satisfy this use case ?
Cheers,
-Tristan
I think that this is an important conversation to have, and I think that many of the considerations are not specific to IDEs. Even if I am using a command line editor and a command line debugger I still want to be able to fetch the source for an element that I am interested in debugging, run the debugger and have it find source files and I want to be able to make a trivial fix and be back at that point in my debugger with the minimum of delay. If we solve these workflows then it should be possible for most IDEs to be configured to take advantage of the solutions. Charles.
Attachment:
signature.asc
Description: PGP signature