Re: [Summary] Meta-data/filesystem-encapsulation

[Kevin Littlejohn <darius connect com au>]

> 
> On Tue, 18 Aug 1998, Kevin Littlejohn wrote:
> 
> > doesn't actually hold out - there are many times when preloading libraries
> > just isn't feasible (ranging from not having root on the box, and root
> > moving your files, to various daemons which may suffer under the added
> > load of these sorts of data checks on files but which still move files
> 
> ... but you don't need root.  You can put this in your .profile script.
> The daemons may not need it.  You're thinking of syslog & co, I presume?
> The metadata will always be independent the file data, whether it's
> embedded in the filesystem or not.  Syslog can append without trashing
> this data.  (And actually, if the data can be embedded, a 'mv' would
> preserve it as well - as long as it stayed in the filesytem.)

What about the case of daemons rotating logfiles?  Or daemons that create
many files (incoming ftp server, perhaps *shrug*).  I want to be able to
assign a particular icon to new, incoming files off that ftp server - but
I don't have complete control over the environment the ftp server runs
from.  I also don't fancy, even on the boxes I _do_ have root on, having
to preload this library all the way through - if my desktop database breaks,
my OS might not boot up?  That doesn't seem too logical to me...

I'm also opposed to embedding information into the files, for a number of
reasons - it's difficult to clean out if I decide I don't like gnome
anymore, it runs the risk of stuffing other non-gnome-aware programs up
if gnome guesses the filetype wrongly, and it's just plain too much tinkering.
It also shares a problem with the next option - how do I assign metadata
to files over which I have read-only permissions?  'cause I'm gonna want
to do that _a lot_ - imagine icons for a shared picture space, or the
ubiquitous methods for log files.

Which leaves the extended attributes - the other case, which only applies
on certain filesystems, and suffers the above file permissions problems.

> 
> > Once you strip the 'but we can cover everything with preloads' off, then
> > you're back down to creating a library (preloaded or not) which is
> > valid for gnome apps (and maybe some other apps, but not all of them)
> > which provides the ability to check 'metadata' for methods on opening,
> > displaying, editing, showing an icon, moving, whatever.
> 
> *I* never suggested that a preload alone would suffice in all cases.  I
> explicitly said (several times) that an alternate would always be needed.
> Preloads simply make it harder to unintentionally orphan data, which I
> think is a Good Thing (tm).

The problem is, once you accept preloads, you start bringing embedded
metadata storage into the picture (under the assumption that everything
will be able to cope, because everything will use the library).  If everything
is _not_ using the library, then embedding information is dangerous -
and if that's the case, the preload scenario _doesn't_ gain you anything
_in terms of design_.  Once we've got a core library, then you could easily
produce a preload-capable wrapper, and preload to your hearts' content on
your own system - but if the design of the metainfo database assumes that
we can gain this sort of coverage, we've got problems.

> 
> > My contention is still that, under the understanding that you _can't_ cover
> > all possibilities with preloads, designing the library to be preloaded
> > gains you nothing - designing a multi-purpose 'metadata' library that
> > can be linked into gnome apps is, on the other hand, a good thing, and
> > if people are really rabid, _that_ can be wrapped in such a way as to
> > preload it into other apps on your own system.
> 
> There will need to be a 'metadata' API, and it is best to keep it isolated
> from the "raw data" API (fread(), fwrite(), etc).  Keeping these calls
> atomic, or allowing them to be atomic, makes creating a preload library
> almost trivial.  That is to say, if you want to replace fwrite() with
> GNOME_fwrite_with_metadata(), that's not a problem, as long as you can
> break it down to two distict functions: [standard] fwrite() and
> GNOME_write_metadata().  If you can do that, you can simply make the
> metadata call the first call made by a fwrite() wrapper that would go into
> a preload library (which would of course be staticly linked).

*nod* We're in agreement here - the only thing that's keeping this running
is the reference to preload, which I think clouds the issue of database
design.  Consider preload as an added extra that might appear sometime
further down the track as a wrapper, and we're both happy :)

> 
> There is one problem that I can think of offhand - you can't staticly link
> executables under Solaris if they use name services, and I haven't found a
> way to make gcc/ld staticly link some libraries but not all.  A simple
> mixture of -static and -dynamic arguments doesn't (or didn't at the time)
> seem to work.
> 
> These are all trivialities, though...
> 
> > The other issue, then, is where you store the data.
> 
> As long as it's abstracted, it is good.  =)

Almost - I think this is the next discussion - my preference is heavily
with a personal database + system database, rather than trying to get the
metadata 'near' the file - I _hate_ having .* files everywhere, and I
think where you put the data is actually near-irrelevant AFA how likely
it is to be orphaned - if you use the metainfo library, it won't be orphaned,
if you don't, it may be...
I also think you dodge the issues of 'whose metadata is it', and 'can I
assign metadata to a file I can't write'.  The _big_ problem is, how do you
export that data to other systems?  Maybe we need a 'gnome-file' type, that
contains meta-info in a wrapper around the file itself... *shrug*

KevinL