Re: [RFC/PATCH] Nonotify - A simplistic way to determine directory content changes

[Alexander Larsson <alexl redhat com>]

On Thu, 2004-06-03 at 14:24, nf wrote:
> On Tue, 2004-06-01 at 12:06, Alexander Larsson wrote: 
> > On Mon, 2004-05-31 at 20:13, nf wrote:
> > > I have posted this to the kernel list. Might be interesting for the nautilus 
> > > developers as well. Perhaps this could be used by "fam" instead of dnotify. 
> > > Cause i don't see any problems with delivering events from a "polling" mechanism.
> > > But i don't know if we really need fam in this case, or rather use it from 
> > > gnome-vfs directly.
> > 
> > Not that this might not be a good idea, but your measurements are a bit
> > skewed. I'm pretty sure that events could be done in a way that uses
> > less resources (and doesn't block unmount), and what you're measuring is
> > the worst-case for a notify approach.
> 
> When you write a file the regular way (not with "echo's", which
> open/closes the file for every line), fam/dnotify "only" slows
> file-writing down by about factor 2. Test-utility attached.
>
> Alexander, another question: Do you think i should continue developing
> nonotify? I have the feeling that anything less sophisticated than
> dnotify&events won't get accepted anyway. 

I like the basic idea of nonotify. Its a lot easier to do resource
control like throttling in the polling daemon, and the kernel complexity
is a lot lower. A daemon using dnotify will need to do full stat-walking
and keep stat struct around anyway, so there is no added complexity on
the userspace side. The only thing that would avoid that on the
userspace side would be full information on changes in events, and that
just isn't workable in kernel-space. 

Two problems from dnotify are still existing, namely notify on NFS and
hard links. The nonotify approach does make the hard links problem
slightly easier, you just have to find all the currently live dentries
that has the inode of the changed file as a child, however I think even
that is not possible. 

It *is* however possible to set the mtime of all the parent dentries of
the inode by walking the dentry->d_inode->i_dentry list, and by doing
that you will make the hard links case work in cases where another
dentry refering to the inode is in memory already. This will tend to
often work, since if fam was monitoring the other dentry, it typically
has stat:ed the file and chances are that its dentry is still in memory.
Its not a perfect solution though.

Anyway, I like the approach and I think it has a higher chance of
getting past the kernel people. I think you should continue working on
it. I think the most important thing at this point is to try to get
feedback and buy-in from the important kernel people. I'll try to point
a couple of them your way if I can.

> The problem is how to bring the dcontents_mtime timestamp to userspace.
> I guess it's hard to get an ok by the kernel-people for a new function
> "sys_nonotify_stat()" in the syscall-table (to report atime, mtime,
> ctime and dcontents_mtime).
> 
> Another option would be to open up a device and use an ioctl call - but
> i don't know if the performance of that is good enough for nonotify's
> purpose. Or add a field to the stat64 structure - which would break C
> compatibility. :-(

The kernel people seem to dislike this approach as "ugly", so I don't
think so. Maybe you could do an ioctl on the fd you get from opening the
actual directory?

> Thanks, and sorry for beeing a bit unkind and provoking in my last mail.

Yeah, being provocative rarely helps your cause, it just tends to make
people ignore you. Anyway, thanks for working on this (long standing)
problem and actually thinking out of the box.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
 Alexander Larsson                                            Red Hat, Inc 
                   alexl redhat com    alla lysator liu se 
He's an unconventional native American photographer with a mysterious suitcase 
handcuffed to his arm. She's a green-fingered motormouth mermaid from out of 
town. They fight crime!