Re: external dependencies

[David Zeuthen <david fubar dk>]

Hi Elijah,

On Tue, 2006-09-12 at 23:15 -0600, Elijah Newren wrote:
> > On Tue, 2006-09-12 at 21:06 -0600, Elijah Newren wrote:
> > > It is much like glibc and Xorg, but there is a bit of a difference
> > > here.  With hal 0.5.8, you're depending on a version of udev so new
> > > that no recent distro releases have it[1],
> >
> > I hear what you're saying, but actually the dependency on udev for the
> > latest release is on version 082 which was released before February 27
> > 2006, e.g. six months ago.
> 
> ...AND didn't do the appropriate fact checking.  Doh!

Actually I was wrong too. Sorry about that. The minimum required version
of udev for the current HAL version (HEAD) appears to be 089 which was
released April 3, 2006, e.g. some 5 months. 

Note that we erroneously required udev 094 (from June 12, 2006), this is
fixed now in HEAD of HAL and I'll be putting out a 0.5.8.1 release with
this and a few other fixes shortly.

A more interesting dependency of HAL, on Linux, appears to be the kernel
version. Since this is not a build time dependency and we don't check
the kernel version when building (since it would break build systems
etc.) the user is bound to see strange bugs if they run HAL on an older
kernel. Most things will be work however, but strange bugs appear e.g.
if the user tries to use LUKS encrypted media from e.g. Nautilus through
gnome-mount or whatever.

I don't really know what to do about this, again, to me at least the
Linux desktop is a really moving target and bugs are fixed each and
every day. The only sane thing to do, at least that what I think (but
then again I'm the bastard known as 'me'!), is to just require the users
to upgrade. That's why distros shipping HAL have the appropriate
Requires: tag in the RPM spec files or whatever packaging system they
use.

> > Unfortunately this isn't possible as some of the features we want to
> > implement depend on bug fixes and newly added infrastructure in the
> > kernel. It's the only way we can sanely move forward without having tons
> > of codepaths and not spend our time supporting bad old versions of our
> > dependencies. The problem space HAL tries to solve just isn't as
> > confined, well-defined or mature as either X.org or the kernel.
> 
> Yep, that makes sense.  As I mentioned before, depending on
> distributor versions of dbus and hal almost certainly isn't feasible
> yet, I'm just hoping to move somewhere in that direction.  It makes
> sense that we're going to have to strike a balance somewhere
> inbetween, and that balance may well be more on the side of the
> bleeding edge right now.  (Note, though, that there is also the
> possibility of making Gnome modules optionally depend on newer
> features of external dependencies instead of having hard dependencies
> on the newer versions, so we have a bit more leeway than either-or.)

It's really what audience we want to optimize for. There are (at least)
two groups of entities that provide the GNOME experience to users; 

 - Distributions - these are likely to ship bleeding edge versions
   of the HAL deps such as Linux or udev anyway. Not so much because
   a new version of HAL requires it, much more because so many bugs
   are fixed with later kernel revisions.

 - jhbuild / GARNOME - users are likely to have old versions of the
   HAL dependencies such as udev, kernel installed.

If we want to optimize for distributors we say, for example, that it's
OK for GNOME 2.18 to depend on the HAL 0.5.9 I will release in December.

Example: with some luck / good timing etc. we can put code in
gnome-vfs / g-v-m for 2.18 to take advantage of new features in HAL
0.5.9 - for example, one planned feature that is planned is making
hotplug for LVM and RAID "Just Work". E.g. you plug in two USB or
Firewire disks that are striped or mirrored and it automounts the file
systems on the disk. 

While some people say this feature might be useless and geeky (hi
Bryan!), some other people consider sane handling of e.g. RAID a
checklist item for e.g. media verticals. Indulge me and think about
photographers for a while. Actually, as I'm into photography myself I've
found that it's not uncommon for professional photographers to have 10+
Firewire disks chained together to access their photos. Look at for
example this guy http://www.luminous-landscape.com/ - anyway, that's one
reason I'm doing this feature, my own photo library is just huge these
days.

If we decide to optimize for jhbuild / GARNOME, this feature will only
land in GNOME 2.20. Distributors, at least the ones with resident HAL /
GNOME hackers, will probably just patch their GNOME 2.18 to include this
feature. And that leads us down the road with lots of vendor patches,
not good, pretty sure we don't want to go there.

This of course is only one example. I submit this is a trade off and it
really depends on who we want to optimize for. To me the choice isn't
that hard to make, but then again, I work for a distributor so obviously
I'm biased.

You know... maybe this just proves that Havoc is right - maybe we're
going through an identity crisis as we're trying to please a lot of
people instead of having multiple efforts with razor sharp focus. I
don't have any good solution to this. Or maybe I'm just having a bad day
for bringing this up, sorry..

> > Now, I can't say what GNOME should do, but if we were to lock down the
> > HAL version we use before way before a release there's really little
> > incentive for me and possibly others to hack on software in that
> > release. It would then land early in the next release, not exactly
> > exciting and an incentive as I would need to keep in mind what features
> > I can and cannot depend on.
> 
> I'm not proposing that external dependencies be considered entirely
> locked down at the beginning of a release; in fact, I wouldn't mind
> dependencies being updated several times throughout a cycle--if
> merited.  All I'm proposing right now is that bumping dependency
> versions should first come with a proposal to the community weighing
> the pros & cons & impact, and that sufficient time is provided to try
> to avoid introducing new build issues where possible.  Further, I
> think such proposals should be allowed at any time through the cycle
> (though there should be an exceptionally strong case to update a
> dependency during e.g. hard code freeze).

Yes, it makes sense to at least record somewhere what the dependencies
are. I do that for HAL, in the NEWS file, see

 http://gitweb.freedesktop.org/?p=hal.git;a=blob;hb=HEAD;f=NEWS

KDE seems to do this right

 http://kde.org/info/requirements/3.5.php

though personally I find such a list way too detailed (like most of KDE,
but that's another thing) but maybe that's just me. Maybe GNOME should
do something like this too? 

(apologies to the release team if you already do this and I'm too big a
doofus to actually find the page.)

> > Finally, the HAL dependency in GNOME is optional. I don't think it's
> > really fair to require the same level of stability as we do for glibc
> > and X.org. Plus with one or two very minor exceptions (edge cases) we've
> > kept the ABI/API since, huh, like 0.5.0 which was released 18 months ago
> > (in so far the so name of libhal as not changed).
> 
> Agreed; I do hope things will move in that direction, though.  D-Bus
> is in a similar position, though I think it is probably closer to
> being able to provide such a level of stability.

Right, D-Bus should hit 1.0 sometime soon. 

(Don't ask me about when HAL is going 1.0, I don't have a good answer at
this point - but if cornered I will think hard about what 1.0 means and
when we can do it. It's just pretty complex to answer this and I'd
rather write code than try to think about that.)

> > > Sep 11 21:39:39 <jamesh>        sure.  But he could have left the
> > > cut-n-paste libvolumeid in there a little longer (preferring a system
> > > version if found)
> >
> > Not to flame or anything, but we announced many many months (more than
> > 6) in advance that it would go from the HAL tree. I'd blame my distro
> > for not catching up / paying enough attention. I also don't see why
> > GARNOME / jhbuild can't simply include a versioned udev dep that only
> > provides libvolume_id. It's not like that would be rocket science.
> 
> Noted; we need to work on that so that we can depend on HAL 0.5.8.  :-)

You want HAL 0.5.8.1 at least as that fixes a few obvious bugs and
relaxes the version of udev required. It should be out in a day or two,
pending some more real-world testing. When I put HAL 0.5.9 out I'll mail
desktop-devel and ask if it's OK to bump the version :-)

Actually I think one answer to this whole mess might be to make
GARNOME / jhbuild just use the OS provided versions of HAL and D-BUS.
Each module that happens to use HAL would have it's own minimum version
of HAL required and if that version is not met then it either

 1) builds without HAL support;
 2) refuse to build

and ditto for other non-GNOME deps such as Avahi, CUPS and possibly even
Mozilla, X.org and so forth. Certainly jhbuild and GARNOME ought to be
able to do that, yes? 

I think this is the best compromise, I'm curious what others think.

Cheers,
David