Re: [Tracker] Indexing removable devices on demand

[Martyn Russell <martyn lanedo com>]

On 05/11/2012 06:00 PM, Felipe Borges wrote:
> Hello everyone!

Hello Felipe,

> I'm a GSoC student working on Documents[0] in order to make it able to
> handle/manage removable-devices[1]. As a matter of fact, Documents uses
> Tracker to read and store information about documents on the system.
>
> My mentor has pointed me that Tracker uses a GSetting to flag whether
> removable-devices are indexable. Since most distros ships Tracker with
> this GSetting toggled "off", and it's not desirable to have Tracker

The default setting is off. This is intended. The reason for this is 
that heuristics around what constitutes a "removable device" are not 
always well defined and it can depend on packages installed (e.g. we've 
seen interesting results when GVFS is not installed).

In the end, we saw MORE cases of removable devices being indexed which 
users didn't want than of those that did. In fact, I don't recall anyone 
asking for removable devices to be indexed as standard for a while now.

As far as Tracker is concerned, there is support for handling removable 
devices uniquely. That is, we associate all files on that removable 
device with a data source in Tracker and use a tracker:available 
property to distinguish between files that are available *now* and those 
that are not (perhaps because the device was unmounted). We also have 
in-house cleaning of the database when the device is not mounted after 
some days (also in the preferences).

If you want any information about this, ask on IRC or here for details.

> indexing removable devices all the time, I would like to work on Tracker
> to make it capable of indexing removable devices on an application basis.

Great! I think this is a much better approach. The main downside here is 
that you may have to wait a little longer for the data to be ready, but 
you have that already with larger datasets on removable media.

> As far as I am concerned, Tracker operates by monitoring and indexing
> files on specific places. In summary, we want to enhance Tracker's API
> to provide what is necessary to tell Tracker when and what to index.

Interesting. Currently, that's possible using tracker-control -f $file 
which in turn uses a D-Bus API provided by tracker-miner-fs. I $file 
might be able to be a directory, I don't remember. This API uses a 
priority queue internally, so if some thing’s being indexed already, it 
should take priority.

What isn't available on demand right now, is removing a file. You would 
need to do that tracker-sparql manually. There are plenty of examples of 
this in tracker-miner-fs.

The *when* part will likely need new APIs (unless you're planning on 
scheduling that yourselves from gnome-documents?).

> I have written this e-mail because I want to hear from you how desired
> this feature is and suggestions on how it should be implemented.

I think it's a good idea.

Right now a lot of the things people are not happy when it comes to the 
right content being indexed are due to us second guessing behaviour from 
users (e.g. indexing the Downloads/ directory when the Linux kernel is 
stored 20 times there... OR assuming all files are in XDG locations 
recursively / just $HOME one level deep). We can't be sure that 
satisfies all use cases.

How do you intend to do this? Ask the user if they have content they 
want indexed on the media and then fire off an API call?

--
Regards,
Martyn

Founder and CEO of Lanedo GmbH.