Re: Thumbnailing service project; opinions, suggestions?

[Rob Taylor <rob taylor codethink co uk>]

Hi Mark,

We already have a thumbnailing service that is only just now starting to
be used. A complete reimplementation of soemthing existing will not be a
very useful thing. I also suggest that 13weeks for your entire plan is
very over optimistic. I would suggest a better option would be to do a
library with optimised thumbnailing routines. This would also be very
useful for existng projects and probably result in you code being used
in a number of situations. To add complexity you could also branch out
into making thumbnail clips for videos. If you write up such a plan
completely, i think you'd find it's more than enough work for a 13 week
project.

Thanks,
Rob

Mark wrote:
> Hi,
> 
> on school i study Computer Science. That exists of a dozen "Major"
> semesters and a few "Minor" semesters. Each lasts 10 weeks and that
> for 3 years.
> So this project, described in depth below, will have to be done within
> 10 school weeks of which the first 8 are the "creation" weeks. After
> that it gets rated by my teachers and i either pass or fail it.
> Now on school i want to learn the following coding methods in a Minor
> part. A minor is a part where i "can" compose my own semester and i
> did it with the following "demands" as in methods i want to learn in
> that semester:
> 
> •       Plugin architecture
> •       Asynchronous
> •       Multi-threaded
> •       Design patterns
> •       Data structures (hash, queue, stack etc...)
> 
> And the methods that will be in it but already "known" to me
> 
> •       Daemon
> •       IPC (D-Bus)
> 
> Now i've read the wish on this list to make a thumbnailing service for
> a generic central way to generate thumbnails.
> All of the above methods can be used in a thumbnailing service. So, i
> made a proposal on school to make just that, a thumbnailing service
> with a plugin architecture.
> 
> Is this been done before?
> -------------------------
> Well, not done before but a group of people is doing it as we speak:
> http://live.gnome.org/ThumbnailerSpec ... i mailed them roughly this
> mail a week ago to see if we could work together on this but i sadly
> had to conclude that their project is in the final stages before release
> 1 so there would only be documenting and bug fixing. That is also part
> of my semester but not the only part. I want to learn more then just
> bug hunting and documentation stuff and for that there is no room in
> their project thus i ended up concluding that i can't join them for the
> simple fact that my school would probably not allow that as a project.
> 
> So, in roughly 13 weeks time (3 weeks in the current semester then the
> 10 weeks for this project) the Gnome community will have the ability to
> choose between 2 thumbnailing services. That is not a maybe but a
> certainty. Why? Well, i made this proposal on school, it got approved so
> now i simply have to make it in order to pass the semester.
> 
> Will i be compatible with their dbus thumbnailing spec? Perhaps..
> perhaps not. I do need to design the project before i start making the
> actual code (major part of the project) but i do not know if i will
> stick to the dbus rules they made. I can see some problems coming there
> if you have huge folders to thumbnail that flood dbus.. O well, time
> tell the answer to this.
> 
> Thumbnailing service
> -------------------------
> The idea in short is:
> The daemon holds the plugin framework and knows how to handle certain
> things (to be determined) defined by class templates and interfaces.
> A plugin can then for example say something like this (in readable
> language instead of code):
> "Hey, i'm the FFMPEG plugin and i can handle the following extensions
> <<long list of extensions>>"
> Plugins should be placed in a folder after which a daemon update
> (through dbus?) should be executed to load and activate the plugins.
> 
> So, what you get with this is a thumbnailing service that has plugins
> (the pure service only holds the plugin architecture and lists from
> extensions to plugins provided by the plugins).
> That also means there will be a pixbuf plugin to read the default glib
> images. (more on this one later after optimizing)
> 
> Another thing which i see as needed but is not part of this school
> project is making a allowed/denied list in the daemon that contains
> the extensions allowed to thumbnail besides the extensions available to
> make thumbnails from. The idea is that you tell the thumbnailing daemon
> through dbus or by reading gconf keys which extensions are allowed
> (or not allowed).
> 
> Then for the dbus part. The idea is to let a application (for example
> nautilus) request the thumbnails from their full scale file paths.
> then the daemon should return the new file path to the thumbnails. That
> is one way but could potential be a issue with folders that contain
> thousands of files.
> 
> Another way to do this is: let nautilus send the folder that needs to
> be thumbnailed. then the daemon should read the folder, check if a
> thumbnail already exists and make it if not. The daemon should then
> (as long as it's busy) send updates (let’s say every 0.1 second) to
> nautilus with the files that have been thumbnailed.
> 
> What probably might be a better idea is to let nautilus look in the
> folder and see which files need to be thumbnailed instead of handling
> that task to the daemon. Then nautilus can send the list of files that
> need to be thumbnailed to the daemon.
> 
> Yet another possible way where i have no clue if it's possible is
> making a file list hash (or map or vector or. anything) in nautilus that
> contains the files that needs to be thumbnailed. Nautilus should filter
> out the ones that are already thumbnailed. Then the pointer of that hash
> (let’s call it a hash for now) should be send to the thumbnailing daemon
> where the thumbnails are being made and returned in another hash
> (also provided by nautilus). I have no idea if this is for one even
> possible and if it's wise to just pointers from one app to another.
> If this gets to complex for the scope of this school project then the
> simple solution (aka the first method) will be used.
> 
> The intention (and requirement) is to make this daemon asynchronous so
> it won't block if another application suddenly wants to request
> thumbnails.
> 
> Optimizing
> -------------------------
> Since the project didn't seem complex enough on paper (can you believe
> that!) i was forced to add one more thing. Luckily i already wanted to
> do something with optimizing in Gnome/glib so i added it to the paper.
> 
> The idea with the optimizing is making the thumbnail generation as
> fast as possible! If you've been reading the gtk devel list lately you
> must have noticed that i already did quite a lot to make thumbnail
> generation as fast as possible. The latest benchmark so far:
> http://img36.imageshack.us/i/scalingperformance.png/
> which shows a significant speed improvement when using threaded
> thumbnailing. This mean that the intended optimizing part is already
> partly done but could be improved more by using threaded thumbnailing
> with gnome_thumbnail_scale_down_pixbuff in combination with
> gdk_pixbuff_new_from_file_at_scale.
> 
> About the image; with the 70 seconds bar my cpu cores (4) ran at 40%
> each and i had "50%" iowait in iostat. The 21 seconds is when the
> images are cached then all cores ran at 100% resulting at 21 seconds.
> The benchmarks are all done with the bilinear flag set where possible
> for the best quality thumbnails.
> 
> When that's working in the daemon it might be possible to speed up the
> rendering of the thumbnails in nautilus (the gtk part of it) by batch
> rendering the available thumbnails if that's not done already. Let’s say
> a nautilus redraw every 0.2 seconds till all thumbnails are visible.
> 
> There are more optimiz ways like using OpenCL or Cairo! more about them
> just below.
> 
> GdkPixbuf? Cairo? OpenCL?
> -------------------------
> First GdkPixbuf. I have been told that the current GdkPixbuf "sucks" and
> should probably be avoided when attempting to make thumbnails as fast
> as possible. One of the reasons to avoid it where Cairo. If i wanted to
> have hardware accelerated downscaling i first need to make a Cairo
> object and from there i need to make a Xlib object. Then i can save it
> with cairo but only in .png and i personally want to be able to pick
> JPEG2000 and JPEG as well. Now i have been told that only the steps of
> converting a GdkPixbuf to a Xlib object could take so long that there is
> hardly speed to gain there. So for that i would have to avoid GdkPixbuf
> and take a more direct aproach. Note that this does fall outside the
> scope of this school project but it would just be nice to have.
> 
> As for OpenCl. I am havily in favor of that method but my hardware
> configuration is sadly the most horrible one to choose. Intel CPU
> and nvidia GPU. there are no stable releases for the GPU yet on linux
> and for the CPU there isn't a driver so i have no OpenCL options.
> However if this becomes possible during my project i will certainly
> attempt to implement it.
> 
> Repository?
> -------------------------
> I would love to have a repository for this on the Gnome servers. Would
> that be possible? it would be my first ever GIT repo. If this is not
> going to be allowed then it's either going to be a github, gitorious or
> google code repository. The code will be publicly available with the
> GPL (v2 or v3) license.
> 
> Blog or Log?
> -------------------------
> I also would like to make daily or weekly posts of the progress. I can
> do that on my own blog but would it then be allowed that my blog is
> readable on planet gtk or planet gnome (or both)?
> 
> Gnome subdomain?
> -------------------------
> If the Repository can't be with gnome i will probably just use my site
> whatever those other 3 repo providers offer. But i would like to have
> a gnome sub domain for example: thumbnailing-service.gnome.org or some
> other name (i still have to make up a name for this project).
> 
> Conclusion
> -------------------------
> As said there was the wish to make this. I made this proposal on
> school and it got accepted. This means i __have__ to make it now.
> That's a risk but in the worst case scenario i just gain knowledge with
> the methods above. In reality i hope this to be useful and perhaps even
> viable for inclusion in Gnome. It might result in a rewrite to be good
> enough for gnome ^_^. My intention certainly is to make it usable in
> gnome whether that is as just a app of mine or a app distributed with
> every new gnome release.. the latter option would be the best one.
> 
> The approved school document
> -------------------------
> I sadly can't give it. It's written in dutch. If you are dutch or want
> to translate it to english you can get it. It's 12 pages of which 3 are
> scheduling pages which can be trashed. The text in this mail describes
> it more in depth then the approved document since that needed to be
> "general".
> 
> Feedback
> -------------------------
> I hope some people could post their opinions about this, where can
> more optimizing things be done that fall in this scope, did i miss
> features for the thumbnialing service? etc.. etc.. Your feedback would
> be greatly appreciated.
> 
> Thanx a lot for reading this long mail,
> Mark
> _______________________________________________
> gtk-devel-list mailing list
> gtk-devel-list gnome org
> http://mail.gnome.org/mailman/listinfo/gtk-devel-list


-- 
Rob Taylor, Codethink Ltd. - http://codethink.co.uk