Re: Finding and Reminding, tech issues, 3.0 and beyond

[Alexander Larsson <alexl redhat com>]

On Wed, 2010-04-14 at 18:04 +0100, Martyn Russell wrote:
> On 14/04/10 16:10, Alexander Larsson wrote:
> > On Fri, 2010-04-09 at 18:09 -0400, Owen Taylor wrote:
> >
> >> "User defined tags"
> >>
> >>    A completely flat view of all documents doesn't handle all users
> >>    or use cases. "Frequent filers" will want to be able to identify
> >>    projects and other subsets of files.
> >>
> >>    There's not a detailed plan for the user interface right now, but
> >>    technically this could be done a couple of ways.
> >>
> >>    We could use the traditional method of grouping by using
> >>    folders; and just make that look somewhat tag-like in the
> >>    UI. (Make selecting a folder show all the files in that folder
> >>    and all sub-folders. Allow creating a folder of files without
> >>    worrying where it was and automatically creating it in
> >>    ~/Documents.)
> >>
> >>    Or we could use a real tag-based approach with tags stored in
> >>    metadata. (multiple tags per file, tags orthogonal to folders.)
> >
> > Does tracker currently index gvfs/gio metadata?
> 
> No.

Since all writes to metadata goes through a centralized daemon it should
be very easy to tell tracker about changes, and the format is designed
so it should be very efficient to index. I've always planned to add
support for this, but its not yet happened.

> That sounds quite similar to what Tracker does, perhaps not as efficient 
> of course (due to infrastructure/IPC/etc).

GVfs metadata is also structured to be very efficient for the typical
access patterns of file i/o (enumerated all files in a directory with
metadata, most consecutive i/o done in the same directory, etc) by
grouping per-directory data together, using mmap to allow efficient
in-process "caching" of data, allows sync operations that are efficient,
etc. This leads to very high performance for the typical gio operations.
Tracker instead has a more general database which is much better for
more complex lookups (like searches or reverse indexing) but is overkill
for directory structured accesses.

> What about files which are not copied/moved/etc with GIO APIs/commands?

Obviously the metadata won't follow these changes, but thats the case
with any lookaside storage for metadata, and even for xattr for any
operation but "rename on same filesystem". Even a "save new copy of doc"
operation will lose xattrs if done using write-to-temp-and-rename-over
if the saving code doesn't do extra manual work to keep them. And xattrs
are still prohibitly slow, see the comments in the blog post i linked
to.