Re: RFC: GNOME 2.0 Multimedia strategy

[jg pa dec com (Jim Gettys)]

I believe we want a (relatively) simple audio server, that exposes most
things to clients (and can be audited for network intrusions).

The issue is we don't have a good one.  We have at least 4 not so good
ones.  And it needs to be satisfactory to not just Gnome, but KDE and others.
(thank you very much: I want to be able to have both Gnome and KDE apps
playing audio on my desktop at once).

These include:
	esd
	NAS
	AF
	A nameless one Keith Packard wrote.

And there are others I can name, if I scratch my head for a while.  Each 
is deficient in some way or another; some more deficient than others. 

Such a server needs to be coordinated with the X server using the X
syncronization extension.

To try to solve all the world's problems (and world hunger), for multimedia 
in one place stikes me as a violation of modularity.  No matter how hard 
you try, you'll not get it right.  I can tell you (first hand - I am coauthor 
of AF, one of the inadaquate servers we have available), that just doing 
the audio server by itself is enough to keep in one's brain at once, without 
worrying about all of multimedia streaming requirements.

So I believe strongly we need to keep separate the signal processing and 
internal needs  (including synchronization) of an application mostly separate 
from the problem of mixing audio streams from different sources 
(applications).  If you do this right (allow explicit control of time, 
as AF does), multiple applications can synchronize quite nicely on the 
rare occasions that they need to.

So lets see if we can get two threads of discussion going here:
	1) an audio server, to deal with the multiple client and hardware abstraction
	problem,
	2) the general signal processing/synchronization needs of applications.
The two should only be loosely coupled (making sure 1) provides what 2)
will need to get its job done, and acknowledging that there will, over
the years, be many more instances of 2) than of 1).

So I claim 1) needs to be much more than a Gnome discussion to succeed.

A year or more a ago, I tried to poke people into working on this problem, 
but didn't get critical mass to make progress.  Keith Packard and I set 
up a "soundserver xfree86 org" mailing list to try to get it going, but 
did not really advertize it much: neither of us has had time to do more 
than comment on what little traffic there has been (both of us have built 
audio servers).  What is really needed is a good architect with enough 
time on their hands (which leaves Keith and I out, being too busy already). 
Maybe we should try again to get things moving in this area.  But it really 
needs at least one full time really good person to make good progress, 
I suspect.  There is alot of code available to help the problem (all of 
the above list's code is available), but the design has to be done and 
a real artifcat built.

That being said, gstreamer, from what I can gather, though I haven't studied 
it carefully, looks like a reasonable library to help the apps do what 
they need and is a strong candidate for Gnome's need for 2).  But I don't 
think it, itself should have to deal with the hardware abstraction and 
network issues on top of everything else.  The problems are enough different 
that trying to mix both into one is likely to fail, and won't withstand
the test of time; I can guarantee we'll learn lots more about signal
processing over the next decade.
				- Jim

--
Jim Gettys
Technology and Corporate Development
Compaq Computer Corporation
jg pa dec com