Re: Online Desktop, Tomboy, and user storage

[john carr unrouted co uk]

Anyone would think I wasn't getting up for work in 5 hours.. I probably
won't be :-)

> Hi,
>
> John Stowers wrote:
>  > I tried to raise this point in my previous email. I dont think putting
>  > app specific logic on the server is all that scalable.
>
> This really depends on the goal. The goal I'm working on is not to have
> a backup of all my private data synced among all my home directories,
> though I think that's a cool project too and people are welcome to use
> any part of online.gnome.org that's useful for that project.
>
> What I'm interested in is unique new functionality enabled by a desktop
> that's a window to the Internet (vs. a desktop that is primarily a way
> to edit my private data). So for example, collaboration features are
> important, and web access is important.

The Conduit.net goal that we had was pretty much exactly this. I think
John means that the API for getting data in and out of the server doesn't
have to be different for each application. It should be pretty consistent
and easy for Conduit to support - it just cares about getting data from A
to B.

> A simple example with Tomboy, maybe you want to be able to say "share
> this note with John and Elizabeth" where John and Elizabeth are chosen
> from your social network. The "ACL" on the note is then set so only they
> can access it.
>
> Then maybe I get real-time notification when either of them is looking
> at the note.
>
> App-specific logic is required to make this kind of stuff go, and the
> app-specific logic is useful for Tomboy even if no other app ever has
> it, then useful for the second app to have it even if no third app ever
> has it, and so on, so the scalability to writing N apps doesn't matter
> much, it only has to scale to however many apps we actually want to work
> on that would actually benefit.

I agree that this is not really a Conduit area, though if we are using
HTTP PUSH, it might be worth subscribing to the notifications on behalf of
apps in Conduit.

Again, I don't deny that server code is needed to make this a rich
experience. But the API for getting data in and out can be consistent
across the different apps.

>> I would like it if you provided a lowest common denominator interface.
>> The ability to get a list of object, to put and get objects, and to
>> associate metadata with these objects. It happens that using WebDAV is
>> an already workable way to do this, but if thats not suitable for you
>> then something else would need to be written.
>
> The online.gnome.org server pretty much already supports webdav, the
> only issues are the authentication-related ones I mentioned before. If
> that can be sorted out (it's mostly a client-side webdav library
> problem), then the server is ready to go, modulo whatever bugs we find.
> We could also and perhaps should just set up Apache for this, or just
> require people to use their own server.
>
> That's separate from what I'm talking about working on though. In my
> previous post I talked about the stuff in addition to sync:

If we are talking about getting data in and out, this is important to keep
in mind. I don't know how much flexibility your server has. I presume more
than "Webdav sent me a file and i stuck it on my hard drive"

> http://mail.gnome.org/archives/desktop-devel-list/2007-August/msg00023.html
>
> The server must have an application-specific data model, or it can't
> have any UI.
>
> On the client side we're doing the same, Tomboy needs an
> application-specific data model to do its UI.
>
> This doesn't preclude or rule out a WebDAV facade to the user's data, it
> just means that the server has to be able to use the data in its
> application logic, for example the server might need to be able to
> convert the data to some Java objects that it can work with. That means
> any generic data store will have to have the equivalent of 'schemas' -
> in a WebDAV context, a fixed location for the Tomboy files and knowledge
> of their file format, for example, would be the schema.

I think I touched on this in my other e-mail. The model for Tomboy is the
Note XML format. This can be synced by conduit, and it doesn't matter
whether its WebDAV or whether you bounce it of 5 conduit servers first.
Whether your using Conduit or not using the native XML format as your
format for crossing the interweb seems pretty sane.

> I am all for a private, "just a file tree" data store on a server with
> no application logic; we could have for example one desktop-wide place
> where you set "my private storage location" and GNOME apps could
> integrate with a shell account or webdav share or S3 account or whatever
> individual users might have in order to store data. As I mentioned, the
> online.gnome.org server even has some support for this already (though
> maybe we want to use Apache when possible). I think another task here is
> to have some defined way to split up the DAV tree among apps, so users
> don't have to manually configure a directory for each app to use.

This is fair enough, but its a bit of an aside. Conduit isn't just about
doing backups to a server somewhere. So while we can support this, this is
meant to be Tomboy online.. so i'm not going to say anything about this
bit.

> This may seem like a basic question, but I was trying to ask something
> lower-level, which is just: which bits of Tomboy code would be replaced
> or hooked into by Conduit?

Basically, as I was said at the end of my last e-mail, Conduit was doing
sync before Tomboy. I don't hold any grudges for Tomboy rolling there own,
they were forced to meet the demands of their users and not depend on a
"2nd class citizen". So Conduit duplicates all of Tomboy's sync code.

> Naively, I would expect that Conduit corresponds to the SyncManager in
> Tomboy, which is a piece of code I don't expect to touch at all. So it
> isn't clear to me how Tomboy using Conduit or not would relate to what
> I'm doing. It is probably clearer to you, because you've spent a lot
> more time in these codebases.

You say that sync is a part of the process. Then Conduit should be a part
of the process. Think of conduit as a libsync for the whole desktop. No,
this isn't about syncing to yet another file store. This is about getting
an object out of Tomboy or Evolution or Epiphany and passing it to OGO.

> To sum up the mail here, one implementation requirement for the user
> experience I'd like to try is to keep an app-specific data model that's
> shared between server-side application logic and client-side application
> logic. But the larger goal is a particular user experience, syncing the
> data model is not an end in itself.

While I try not to talk about them (because they are very grand and over
engineered) I have thought about data models and how they can be made
pluggable - enabling you to run F-spot headless and have multiple clients
or other such craziness. But thats besides the point.

What I see is:

Tomboy UI <--> Tomboy Backend <--> Conduit <--> OGO API <--> OGO Backend
<--> OGO Frontend

Here the role of Conduit is to orchestrate working out which data has
changed and move it backwards and forwards. It handles conflict
resolution. The Tomboy DP grabs XML files using Dbus and passes them (only
when needed) to the OGO API which arranges for them to appear in the
online UI.

Maybe my in-experience is showing when I look at sync and think, thats not
a trivial problem to solve... but what i'm suggesting seems like a
worthwhile thing not to have to worry about at this stage.

John C