[Builder] LibIDE Update

[Christian Hergert <christian hergert me>]

Hi Everyone!

Many of you know I've been busy working on LibIDE. It has come a long 
way since I started. I finally have a better understanding of how things 
are going to work going forward.

I'm currently deciding whether or not we should start pushing this into 
what will be 3.16. We have a few weeks left, and to be honest, I think 
it's more important to get people testing this than adhering to code 
freezes.

So, what has been implemented since we last talked?


## Extension Points

LibIDE does not have plugins in the traditional sense. Everything is 
done with `GIOExtensionPoint` and the implementation is compiled inside 
of the library. It's sort of a dependency injection without all the 
brittleness. It works well combined with `GAsyncInitable` to create 
instances without knowing the concrete implemention. It allows the 
implementations to self-test if they can support the requested feature 
with what is loaded in the `IdeContext`.


## Language Abstractions

The language abstractions have landed. You can get an `IdeLangauge` for 
a given `IdeFile` using `ide_file_get_language()`.

We use a combination of `GtkSourceLanguageManager` and 
`GIOExtensionPoint` to load these so that we can provide exceptional 
features for certain languages. For example, C will use an 
implementation named `IdeCLanguage` where as other languages will use 
the generic `IdeLanguage` base class.

The `IdeLanguage` class has various properties that will allow us to 
simplify the editor view in Builder. For example:

 * `ide_language_get_diagnostician()` returns an `IdeDiagnostician` 
which is a collection of `IdeDiagnosticProvider`s. `IdeCLanguage` will 
return a diagnostician with a provider which accesses clang in process. 
We needed to bring clang in process to get some of the interactivity 
performance we want. However, they can also use remote providers, such 
as those coming from `gnome-code-assistance`.

 * `ide_language_get_indenter()` returns an `IdeIdenter` that can be 
attached to a `GtkSourceView` to provide identing features. The 
implementation for these still need to be brought down from 
`GbSourceAutoIndenter`.

 * `ide_language_get_highlighter()` will return a `IdeHighlighter`. 
This isn't a replacement for `GtkSourceView` highlighting, but an 
extension to it. It will provide semantic highlighting of things like 
type names and functions. The `IdeCLanguage` will return an 
`IdeClangHighlighter` that will use the translation units provided via 
`IdeClangService` to extract tokens in the file.

 * `ide_language_get_symbol_resolver()` will return an 
`IdeSymbolResolver`. This can be used to get an `IdeSymbol` for an item 
at a given `IdeSourceLocation`. This will let us also find other 
locations that the symbol exists. (Imagine highlighting all instances of 
a local when it is hovered). We will also have support via this class to 
get all the symbols in a file. That will get used in a symbol list 
popover or something to that nature.

 * `ide_language_get_refactory()` returns an `IdeRefactory`. This is 
still a bit of a gray area as to what will be supported. But the handy 
wavy design is that it will get us access to a series of commands to do 
common features like rename methods or locals. Extract


## Diagnostics

You can test out the new diagnostics engine using 
`./ide-list-diagnostics configure.ac myfile.c`. It is language agnostic, 
so if you use C it will use the C diagnostics provider, and if you use 
Python it will be provided via `gnome-code-assistance`. Either way, you 
get a nice clang style printout on the command line.

  https://pbs.twimg.com/media/B93wC5xCcAEDqKf.png:large


## File Settings

We haven't had a good story for editor settings as of late. Everything 
is backed by GSettings. Clearly not ideal.

So this week I added `IdeFileSettings`. This is an abstract base class 
that can have various implementations, provided via extension points.

The idea is that you can asynchronously load the file settings for an 
`IdeFile` and it will discover the most accurate file settings for the 
file. This might come via modelines (still todo), editorconfig via 
`IdeEditorconfigFileSettings`, or gsettings via `IdeGsettingsFileSettings`.

 `ide_file_get_settings_async()` will perform the operation. You can 
test it with `./ide-list-file-settings configure.ac test.c`. We require 
the `configure.ac` so we can load the project. Not ideal, but that is 
what is there now.


## Search

I've been moving search down into LibIDE. This is going to allow for 
lots of cool things like using the scripting engine to add your own 
search provider. Probably useful if you have some custom bug tracker at 
your company.

You can test the engine with `./ide-search configure.ac [KEYWORDS...]`.


## Building

I started plumbing support for building projects. It's not very far 
along yet, but a basic build command works for the local project. We are 
doing all builds out of tree. (In 
~/.cache/gnome-builder/builds/<project>/<arch>/). The root build 
directory can be changed. But I definitely want to try to do all builds 
out of tree. It really simplfies the problem when compiling for external 
devices.

You can test this out using `./ide-build configure.ac`. It is not good 
about dealing with the situation where the tree has been configured 
locally, but we'll work through that in the not too distant future.


## Project Files

When the `IdeContext` is loaded, the discovered `IdeVcs` will load all 
the project files into the project tree. I went back and forth decided 
if the BuildSystem or the Vcs should load this content, and I think the 
Vcs is the right place to do it.

You can test this from the command line using `./ide-list-files`.


## Device Management

You can fetch the `IdeDeviceManager` from the `IdeContext` using 
`ide_context_get_device_manager()`. The device manager as a series of 
`IdeDeviceProvider`s loaded. When they discover a new device (such as a 
tablet, phone, simulator, etc) they notify the device manager.

When we build, we build for a particular device. The `IdeLocalDevice` is 
probably the most common device we'll be using.

You can list the discovered devices using `./ide-list-devices`.

    $ ./ide-list-devices
    local "starlight" (x86_64-linux-gnu)


## Unsaved Files

One of the things we have wanted for a while is the ability to track 
unsaved buffers and persist them to a drafts directory when closing 
Builder. Support for this is in LibIDE but not yet taken advantage of in 
the Builder UI.

The abstraction also plays nicely with background diagnosticians since 
we need the unsaved buffers constantly to generate accurate warnings and 
errors.

When we load the IdeContext, the unsaved buffers will be brought back 
into memory, and the documents loaded into the document manager (still a 
todo item).


## Code Navigation

I added `IdeBackForwardList` to LibIDE not too long ago. This will serve 
as the navigation stack for browsing through code. Whenever we reach a 
jump location in the editor, we'll push or position onto the stack. 
There are some non-obvious implementation details here to make things 
"feel right". Go check out the code to see how we deal with branching 
and merging histories.


## IDE Scripting

LibIDE gained support for IDE Scripting this week. Currently, we only 
have one scripting language engine, which is GJS based. I bit the bullet 
and had to write some C++ to host the JS runtime, but was pleasently 
surprised how well this works with automake these days. We still have a 
C library with C linker yet some code calls into C++ libraries. Lovely.

So what can you do from the scripting environment? Turns out quite a 
bit. For example, you can implement your own search provider. You could 
also navigate the project tree, extract information from the version 
control system, and before long, register commands.

We'll also make it easy to hook into the file save/load pipeline as soon 
as the document manager is moved into LibIDE.

Garrett Regier, the libpeas mantainer, also started work on a Python 
loader. I expect that will merge into wip/libide relatively soon. This 
is great news from an extensability perspective.

I've tried to be careful about what I've exposed in the LibIDE API. That 
means that many constructors and helper functions are not exposed, but 
kept internal to LibIDE. By doing so, it is pretty clear what you can 
and cannot do from the scripting environment.

-- Christian