Re: Nautilus/Medusa search index enhancements

[Curtis Hovey <sinzui cox net>]

On Wed, 2003-02-19 at 15:42, Manuel Amador (Rudd-O) wrote:
> Curtis Hovey wrote:
> 
> >>Medusa should be signaled by autorun whenever a drive is inserted, or 
> >>by 'dynamic' where a volume is hotplugged - the whole idea being that Medusa 
> >>should detect mounts and act accordingly
> >>Medusa should start indexing files if it's a new volume or begin monitoring 
> >>files there for changes. 
> >>    
> >>
> >
> >We need to reconcile that medusa is now a user application.  Each user
> >would need their own preferences.
> >
> WHAT?  Medusa isn't a search daemon anymore?  Then the whole idea I had 
> crumbles to pieces!
> Who decided Medusa should change into a user-level application?  How are 
> you going to build a NFS server index now?

To answer B first, Rebecca, medusa's maintainer.  Seth helped port it to
GNOME2.  As for A, your plan isn't gone.  Medusa still works, and it is
very fast.  I believe making medusa a user level application makes it
play well on many OSes and simplifies user preferences.

> >I believe it has from the start.  You can configure which file systems
> >to not index.  What it doesn't support it multiple roots for an index,
> >requiring users to create many indexes.
> >
> No way.  The idea was that the user DIDN'T have to configure anything at 
> all.  That the user could rely on Medusa to smartly choose whether to 
> index a remote file server or contact another Medusa server there to 
> return search results.

Well I'd like that, and it can be done.  It's a usability issue
complicated by the user's intent of the index.  I maintain indexes as
root and as a user, and I'm frustrated by managing the paths in my
indexes and what it indexes.  As a user I'm not interested in the noise
in my results from cache/, tmp/, .bak., '~' files, but root is.  I want
to index several remote volumes, but root doesn't.  

I think the problem can be simplified use a preconfigured HOME index for
each user, and an ALL index shared by all users.  Medusa may need a
preferences to set file types and paths to ignore. Or the directory
properties in nautilus, and file types/programs can be extended.

Distributed searches are separate indexes, so it might be easier to
implement your ideas now that medusa knows about multiple indexes.  We
need an intersection/union routine for multiple indexes.  Such a trick
could be further extended to integrate results from Google.

> >>(*) FAM delegates file monitoring to remote FAMs. When FAM cannot connect to a 
> >>remote NFS FAM server, it falls back to standard dnotify. This behavior can be 
> >>mimicked in Medusa-searchd. To prevent failed file accesses, removable media 
> >>search results wouldn't be returned to the client search service.
> >>    
> >>
> >FAM will be useful when preforming incremental indexes.
> >
> Wrong.  FAM is a file notification daemon.  You really expect the search 
> daemon locally to ask FAM to monitor EVERY file in the file server? 
>  That's not possible.

Maybe not, but I'd like to see incremental indexing.  I'll resort to
cron if I have to.

> >>Medusa-searchd in the NFS server should accept remote connections if the NFS 
> >>server is up
> >>    
> >>
> >Medusa-searchd doesn't exist in medusa-2.0.
> >
> Yes.  Now it's useless.

Nautilus doesn't know how to select the index.  My own thoughts were to
have an ALL index owned by system/nobody, and a user index named HOME. 
The user can select the index to search.  The URL RFC need revision. 
I'm thinking the either extending the search method, or adding a new
part to the URL to indicate index.  Querying and mering results from
several indexes can be done, though that will be more difficult.

I prefer using standard names like HOME or ALL so that a URL sent from
one machine to another would be portable.  Another way would be to only
search HOME or ALL.  Nautilus compilation currently stumbling over a
method that returns medusa-searchd's status.  I'm planning to replace
that with a method that checks for the existence of an index.

> >>I have the feeling that a couple of changes in Medusa would render its 
> >>usability much greater than the current prospect. I bet if this gets worked 
> >>upon, even the KDE people would get around to using it. Remember how ugly and 
> >>slow the search box in KDE is. And another thing: it's slower than Windows' 
> >>file search tool. KDE already takes advantage of SGI FAM. Medusa could be the 
> >>search service everyone expected.
> >>    
> >>
> >
> >While that would be great, KDE is prejudiced towards C++.
> >
> So?  Don't we have a library they can wrap?

KDE/GNOME/medusa developers certainly can write a wrapper to access
search, and extend Konquerer to use it.  Gstreamer is in this position
right now.  They have a good low level architecture, and some users are
making KDE interfaces to call it, but KDE isn't willing to accept it as
part of KDE because it isn't native c++.

> >At this time, getting Medusa to work with Nautilus is the first
> >priority--The user level indexes do not work with nautilus, which still
> >looks for the old medusa-searchd.
> >
> Who's in charge of Medusa now?

Rebecca is.  I think I'm the only user looking a medusa code on a weekly
basis.  My priority is to get it working with nautilus, so that other
users can sample it's power.  I need some content indexers, and a better
means controling what and where indexing take place.

-- 
__C U R T I S  C.  H O V E Y____________________
sinzui cox net
Guilty of stealing everything I am.