Re: [Tracker] Proposed future for Tracker - Smaller modules

[Martyn Russell <martyn lanedo com>]

On 08/09/14 17:46, Philip Van Hoof wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 8/09/2014 17:11, Martyn Russell wrote:
>
> Hi Martyn,
>
> > > Without tracker-store, libtracker-sparql can't work.
> >
> > It's more the other way round I had in mind, without
> > libtracker-sparql, tracker-store works.
>
> Actually, libtracker-data (the code of tracker-store) is also the
> direct-access implementation of libtracker-sparql.

Yep.

> The tracker-store itself is a bunch of in Vala written D-Bus service
> wrappers around the same libtracker-data (that is used for
> libtracker-sparql-direct's implementation) plus an eventloop and
> checkpointing plus some initialization code and DBus daemonization.
>
> That's it, that's what tracker-store is :)
>
> Everything is in libtracker-data, which is used by libtracker-sparql

I'm not contesting that, not sure what your point here was here?

> > I wouldn't split the ontologies out, the store is no good without
> > the database and without the schema it's even more useless :)
>
> Sure it is. There are already a few companies that are using Tracker
> with a completely different ontology.
>
> The tracker-ivi project comes to mind. Will run on the dashboards of
> some nice cars.

Right, but then if you want that level of granularity with the 
components, libtracker-sparql should be separate from tracker-store too. 
Since you can perfectly use tracker-store without it.

> > Besides, all the ontology validation and handling is in the
> > libraries the store depends on.
>
> Not correct. All ontology validation rules are in the ontology's own
> introspection. The libtracker-data's ontology validation rules are

Let me elaborate.

I meant validating the ontology schema is correct, before tracker-store 
gets to them, they're just text files, possibly incorrect. Those rules 
(described by the text files) still need to be evaluated.

I also meant reading changes in the ontology and general ontology change 
coping / upgrade path handling.

> Meaning that data/ontologies is completely separate from libtracker-data
>
> You could install a complete different data/ontologies, and
> libtracker-data would just happily crunch that and assembly you a
> different meta.db with different ontology validation rules.

Yes you could. But more importantly you could not use the store without 
an ontology.

> In fact, several embedded-solution companies do this today w. tracker.

Of course.

> > Yea, single point of update.
>
> But whether it comes as a implementation detail of libtracker-sparql
> or is a central process of a Desktop session; shouldn't be of concern
> to whoever links with and uses libtracker-sparql.

I agree, but we're not there yet.

> > > - Providing a ontology
> >
> > Not sure I follow you here, the reason for the store is not to
> > provide an ontology - at least libtracker-data does a lot of that
> > stuff - I would have to double check this.
>
> libtracker-data doesn't provide the ontology, data/ontologies does.

s/ontology/database schema/

> The libtracker-data implementation (which is also used 1 on 1 by
> libtracker-sparql's direct-access implementation) doesn't care for 99%
> what the ontology's content is.

On some level that's not true. Clearly libtracker-data has to interpret 
the ontology to know what rules to follow for database upgrade paths and 
also what's allows or not allowed.

Either way, we're getting off topic here I feel. The point is more about 
components / modules that should fit together or not. The 
libtracker-data library is internal so where it would go is quite 
irrelevant in some ways because it's supposed to be private anyway.

The real question is, should the ontology be separate?

> > > When your process links with libtracker-sparql and uses
> > > direct-access mode, it effectively has everything it needs to
> > > deal with meta.db.
> >
> > Indeed.
>
> Voila :)
>
> So why is tracker-store a separate central desktop-session daemon?
>
> No reason. libtracker-sparql users shouldn't care.

Off the top of my head:

1. To provide a DBus interface (may be a legacy reason, but ...)
2. To provide concurrent connections a way to serialise updates
3. To provide a central and singleton authority to the DB schema
4. Because of database locking mechanisms employed by SQLite?
5. To avoid possible race conditions?

The last two could be wrong, its been a while since I had to care about 
this stuff :)

I'm sure many of those things can be improved these days, but you're 
talking about making this decision upon some utopian idea that is not 
implemented yet. I am talking about doing this for what we have *now*.

> > Consider the application that only wants to query and not update
> > the DB - they don't want to depend on all the crap needed for
> > updating the DB, just the raw libtracker-sparql part.
>
>
> Except that they already and always depend on tracker-store. You can't
> avoid it (read libtracker-sparql's initialization code: you always and
> necessarily depend on it).

I guess it depends on what level you care about "depending" on something.

--
Regards,
Martyn

Founder & Director @ Lanedo GmbH.
http://www.linkedin.com/in/martynrussell