Re: New module proposal: tracker

[Martyn Russell <martyn lanedo com>]

On 29/10/09 15:23, Zeeshan Ali (Khattak) wrote:
> Hi,
>
> On Thu, Oct 29, 2009 at 1:05 PM, Martyn Russell<martyn lanedo com>  wrote:
> > On 28/10/09 23:23, Zeeshan Ali (Khattak) wrote:
> > I think assuming that a project has to be shipped by distros before it can
> > be in GNOME or visa versa doesn't make sense.
> >
> > Distros ALWAYS ship what they want and they want something stable. 0.6 is
> > the last stable version we have and 0.7.
>
>     Distros do ship what they want but there is a predictable pattern
> to their desires. :) What about maemo? N900 will continue to ship with
> 0.6 and if i want my app to work on N900, i'll need to keep on dealing
> with 0.6 for quite some time in future.

There are talks of upgrading it on the n900. Nothing more than that 
right now.

>     Correct me if i am wrong but 0.7 release happened fairly recently,
> no?

Yes, sorry that was a typo, s/and 0.7//.

> Its been quite some time that I've been dealing with crappy 0.6
> APIs so as an application developer I (and others) would need some
> time to evaluate 0.7 APIs if they really are as perfect as you claim
> or not.

The APIs in 0.7 are diminished since 0.6 because we use SPARQL to 
query/update the databases. The benefit of this is that the dbus/library 
APIs don't change. The only exception here is the libtracker-miner API 
which is a new addition and still being improved on in areas. But that's 
more for applications wanting to insert data not query it.

The SPARQL we use changes at times too (if you consider that API) but 
usually to fix or add features that didn't exist before. To give you an 
example, we are considering adding a coalesce function for when data 
doesn't exist to have a default or fallback value. This would mean you 
could use the following SPARQL:

  SELECT COALESCE(nie:title(?n), nfo:fileName(?n), "unknown")

This means, (in the case where a video file has no "title" metadata, the 
title would be used if available and fallback to the file name if it 
wasn't. Of course, if the file name is not available then "unknown" 
would be used as a last resort.

This function doesn't exist yet in Tracker, but to add it just means you 
get more flexibility in your query language. There is no library or DBus 
API that changes here.

> Working with 0.6 I've known nothing but pain (leaving aside
> all the political pressure I've been facing for making my app so
> dependent on Tracker) so please understand that I need some time to be
> sure that 0.7 is completely different in that regard. Until then, I am
> skeptical about Tracker to become integral part of GNOME.

I hear the same thing over and over again from people that used 0.6. 
Carlos and I have spent 2 years refactoring and redesigning the *ENTIRE* 
code base. Not only that Jürg and Philip have also been doing the same 
since they came on the project (Philip in April 2008 and Jürg in 
November 2008). Helping in this have also been Ivan Frade and Mikael 
Ottela (at Nokia) since the project was taken on for the Maemo 5 platform.

During the time that Carlos, Mikael, Ivan and I have been fixing 0.6.x 
issues for the Maemo platform, Philip and Jürg started working on 0.7. 
which drastically changed the design and communication between all major 
parts of Tracker. This came after a lot of review and discussion by the 
core team.

Comparing 0.6. to 0.7. is folly, they are so different and any project 
would be after 2 years of full time coding from 6 people and public 
contributions on top of that.

>    Also I share the concern of Lennart about inotify limitation and
> from my talks with Tracker developers so far, it seems they
> underestimate the importance of fixing this limitation wrt Tracker.

I agree it needs fixing, but there are a number of things to consider here:

 - FANotify is being worked on by Red Hat and will be in the kernel for
   us to use at some point - and we will adopt it then (I believe it
   almost made it into the latest Fedora but didn't so should be in the
   next release)

 - We changed the locations that are indexed by default from $HOME to
   use XDG user dirs for documents, desktop, music, pictures and videos.
   So the focus has changed slightly to the things you most likely want
   indexed instead of EVERYTHING. Of course adding EVERYTHING into the
   config doesn't escape the fact that inotify is limiting us.

 - In the grand scheme of things, I don't think it is that important to
   fix as Lennart says. I don't consider myself a normal user and I
   don't even breach the inotify limit (I come close though with all the
   project sources monitored). I personally would much rather have an
   Tracker which is fast to query/update and has good coverage on the
   metadata it extracts as a priority over supporting EXTREME use cases.

I do want this fixed but it has not been our focus because we have had 
so many other issues to contend with first which were much more important.

--
Regards,
Martyn