An alternative to gdk-pixbuf

[Magnus Bergman <magnus bergman snisurset net>]

Over the years it has been discussed from time to time to replace
gdk-pixbuf with something else[1][2]. Something was even in the making
(I guess over ten years ago) but it never replaced gdk-pixbuf
apparently. Now I don't even remember what it was called. And something
else called pig was proposed more recently.

One major reason to replace gdk-pixbuf has been the long term goal to
phase out GDK in favour of cairo. And some people (including me) has
been displeased with the gdk-pixbuf API for more subtle reasons. It has
also been brough up that it lacks some features (which I guess are hard
to implement as an afterthought). I finally took some time to design an
image loading library on top of cairo called abydos, which at least
suits my needs. And also some needs mentioned in this list over the
years. First I thought it could suit the needs of GTK+ as well. But I
just learned that even cairo is now considered "legacy technology" by
the GTK+ developers[3], so I guess cairo is also being phased out now.
But in favour of what? Maybe the basic design of my API could still be
of some interest. And perhaps I could even port my code over to whatever
is the next hot drawing library.

It's mainly a proof of concept right now. But it works well enough to be
tested. It doesn't have many loaders, but I assume it will be a trivial
task to port them over from gdk-pixbuf since the plugin API:s are quite
similar. The API for handling variants (see below) is work in
progress and not even implemented yet. And I plan to add support for
stereoscopic images as well.


The main features of abydos

* It's easy to use, especially for the common use cases.

* Depends only on cairo. (The main library that is, loaders may of
  course have other dependancies.)

* Selects what loader to use based on MIME types. (But does nothing to
  determine the MIME types by itself.)

* Full support for vector graphics. (Scaling and other transformations
  can be done using cairo.)

* Supports animations, both frame based and continous. (But does
  nothing to measure clock time or schedule events by itself.) Where is
  also no intention to support proper video (streaming) formats.

* Supports layers. As in toggling layers on and off. Just delivering
  each layer as a separate image might not be feasible since a layer
  can be more like a filter operation than a pixmap.

* Supports multi page images. This means image files that contains more
  than one image. Which for example TIFF files can.

* Supports multi variant images. This is somewhat like multi page images
  except that it is known that the images are all (differently sized)
  variants of the same image. It could be embedded thumbnails (which
  could be intelligently cropped and not just scaled down versions of
  the full size image). Or it could be icons with different details
  level for different sizes. Or it could be mipmap textures.

* Deals with non square pixels. By default it scales images to the
  correct aspect ratio, but can optionally deliver them pixel by pixel
  as well.


What it lacks in comparison to gdk-pixbuf

* It (currently) lacks support for saving images. Of course images can
  be saved to PDF, PNG and SVG using cairo. I would like to add for
  some more general purpose animation format to complement those (MNG
  or APNG probably). As far as I know the image saving functionality of
  gdk-pixbuf isn't heavily used. At least not for something other than
  PNG.

* It lacks support for indexed colors, since cairo lacks that support.
  (So this is more of a question about what cairo lacks in comparison
  to GDK.)

* It lacks support for determining the MIME type of files. This is a
  hard problem to solve and I think it's better solved elsewhere. Using
  the shared MIME database for sniffing goes a long way, and is surely
  enough for the needs of GTK+. But not all types can be detected this
  way. Using libextractor (with special plugins) goes a bit further.

* It doesn't use GObject. But adding GObject bindings would be possible
  I guess.


Some examples

This is how to convert an SVG file to a PNG file.

    cairo_surface_t *surface = abydos_load("image/svg+xml",
    "example.svg"); cairo_surface_write_to_png(surface, "example.png");
    cairo_surface_destroy(surface);

This is how to convert the second frame of a GIF animation to PNG.

   abydos_t *ar = abydos_create_from_file("image/gif", "example.gif");
   abydos_set_frame(ar, 1);
   cairo_surface_t *surface = abydos_get_image_surface(ar, 0);
   abydos_destroy(ar);
   cairo_surface_write_to_png(surface, "example.png");
   cairo_surface_destroy(surface);

This is how to display a GIF animation to something cairo can draw on.
Of course some kind of event scheduling should be used insted of a
delay in the real world.

    abydos_t *ar = abydos_create_from_file("image/gif", "example.gif");
    double last_timestamp = now();
    for(;;) {
        abydos_render(ar, cr, 0);
        delay(abydos_get_delay(ar));
        double timestamp = now()
        abydos_add_time(ar,timestamp - last_timestamp);
        last_timestamp = timestamp;
    }


Less is more from a security perspective

It has often been noted that support for more than a core set of image
formats is a severe security risk. One conclusion from this is that the
library GTK+ uses for loading images should only support a core set of
image formats. It has been said that if a dedicated imageviewer wants
to support more formats (which they should think twice about since it's
also a security risk) they should have their own plugin system. But
image viewers generally don't do this. And it would also be a bit
impractical to maintain image loading plugins for severeral image
loaders. I've been in contact with a two creators of image viewers and
they found it unnecessary to have their own plugin system since
gdk-pixbuf already has one. And I think it's a valid point. This
argument could however be nullified by simply disallowing third party
plugins.

What it all comes down to is that using a bigger variety of software
creates a bigger attack surface. Viewing a lot of images is less secure
than never viewing any images at all. But viewing images with a larger
variety of loaders (for the same formats) could also be considered less
secure than using only one loader. So forcing every image viewer to
have their own loaders is also not optimal from a security perspective.
Sharing the same loaders could also be a good thing if get gets more
attention and care. (This is also a good argument against writing yet
another image loading library on the other hand.)

I think a reasonable compromize would be to block all but a core set of
formats (MIME types) in specially vulnerable situations (like for auto
displaying images from internet). And for extra security all but a core
set of loaders could be removed. But this should be done on a distro
level, not enforced by developers, I think.


For more information about abydos, including API documentation, see:
http://snisurset.net/code/abydos/

[1]
https://mail.gnome.org/archives/gtk-devel-list/2007-October/msg00034.html
[2]
https://mail.gnome.org/archives/gtk-devel-list/2014-October/msg00017.html
[3]
https://gitlab.gnome.org/GNOME/gdk-pixbuf/issues/13