Re: FUD about security and file extensions (was Re: Why file content sniffing sucks)

[Rodney Dawes <dobey free fr>]

On Hën , 2003-12-29 at 10:21, Jason A. Pfeil wrote:
> On Fri, 26 Dec 2003, Charles Goodwin wrote:
> [...snip...]
> > Yes, file sniffing is slow.  So implement it in a way that does not
> > affect the user.  Last time I used Nautilus, I could scroll up and down
> > and jump between folders without extra pause, whilst Nautilus updates
> > itself in the background.  So what is the issue?  It only updates what
> > is in immediate view (as I recall) so you just scroll to your desired
> > file and, if necessary, wait the 2s for it to be detected.
> 
> I think that this is how it was supposed to be implemented, but I don't
> believe that this implementation has actually been done.  As a quick test,
> open /usr/bin in nautilus.  On the box I am using to write this, /usr/bin
> has 2,124 files in it.  Nautilus took about 10 seconds to load the
> contents of that directory.  In comparison, use a shell and cd to /usr/bin.
> Then, type the command:
> 
> echo *
> 
> and see how much faster that returns.  echo * returned instantly on my box.
> If you want, you can do the echo * method first just so you can convince
> yourself that nautilus is not caching things for the shell.

And how do I convince myself that echo is doing all of the same
activities that nautilus is? Doing a simple "echo *" has substantially
less traffic to the X server. It doesn't tell you mime types,
modification times, number of files in the child directories, or
anything extra about the files. Hell, it doesn't even put the filenames
on new lines. If you want to make a comparison like this for mime types,
then either compare file and test-mime from the gnome-vfs tests/
directory, on a terminal, while redirecting STDIN and STDOUT to
/dev/null, so that terminal X traffic and rendering speed doesn't come
in to play. Then you should see that it is not the mime type detection
that is slow. The gnome-vfs mime test and file take about the same time
to complete. It took less than half a second for gnome-vfs to get the
mime types of over 619 files on my system.

> Also, when nautilus is opening the directory, take note of the extreme disk
> churning you hear and note its absence when using the echo * command.  FYI,
> the hardware I ran this on is a 1.8GHz P4 with 1GB RAM and U160 SCSI disk
> on an Adaptec 7892A controller.  Hardly pokey by any standards.

Yes. The "echo *" command doesn't open any files at all. It's all done
in the command line shell. It simply expands * to all filenames in the
current directory, and spits them out on STDOUT. It is not doing
anything complex at all. Maybe you should fix The GIMP to use "echo *"
when it starts up. It takes a long time to load all those plug-ins.

> If we want the average user to use Linux over Windows, we have to have a
> system that is competitive not only in security, but also in speed.  While
> nautilus has improved greatly over the years, it is still *far* behind
> explorer in speed and this file-type identification issue is a prime reason.

s/Linux/GNOME/ Linux in general is as fast as, if not faster than,
Windows. Nautilus is also just as fast as Explorer. Saying X is faster
than Y because Y is doing more is ignorant. If Nautilus is slower than
Explorer for you, it is a configuration issue, not an issue of how it
detects the mime types. You really should spend your time profiling the
GNOME libraries and Nautilus if you think your statement is true.
Stating your opinion of what is wrong with it won't get it fixed.
Proving where the speed bottlenecks are with profiling data and real
comparisons with more constants than the hardware it was run on, will
be much more useful.

> > If Nautilus is wrongly detecting a file type it is a _bug_ and should be
> > dealt with as such.  It is nothing to do with the system used by
> > Nautilus.  Detection of type by file extension is far more error prone
> > and relies much more on correctness of user input which is an
> > unreasonable expection on lay users.
> 
> I dispute this notion.  Proper training of users (not unreasonable) with the
> help of applications automatically setting file-types will go a long way
> towards having good input from users.

You can't claim a dispute and then agree. Reasonable training and
application assistance will go a long way, yes. However, we shouldn't be
relying on user-training. We should be writing applications that help
the user to learn on their own. Psychology is a very important aspect to
usability. Having a user do something on their own instills pride. Don't
you feel like doing more when you do something yourself, as opposed to
having to spend $1200 to learn to write a basic web page? Our
applications should be smart enough to let the users know what they are
doing, and if what they are doing is correct. Also, people migrating
from Windows or Mac OS have a bit of training. They generally know
enough about icon/label->action associations and similar, that they
can figure out things fairly quickly.

> [...snip...]
> > > The bugs present in Micros~1 Windows are not due to file type detection
> > > by suffix. 
> > 
> > Wrong, they are.  By due nature of the ridiculous method, people
> > associate .jpg files or .gif files as images.  This introduces a problem
> > with visual association.

> > Somebody gets an email with an attachment such as 'pretty.jpg.exe' or
> > 'sexy.gif.pl' and they open it up.  Yes, this is due to file type
> > detection by suffix because you are subconciously causing people to
> > recognise file types by file suffix and hence they can be easily
> > mislead.

> However, in this type of example, the extensions for both of these files
> are .exe and .pl, respectively.  Therefore there is no ambiguity since a
> file ending in .exe is not a file ending in .jpg and a file ending in .pl
> is not a file ending in .gif.  The only time that this can cause a problem
> is when the system *hides* the extension like Windows does.  This is a
> badly contrived example.

No. It goes beyond hiding the extension. Many versions of the file
system also only support a maximum of one period in the filename. This
really causes problems. The suffix detection also plays a major role,
but it is not the only contributing factor.

> [...snip...]
> > 
> > One goal of Gnome is to make Free Software desktops a global reality (as
> > if it already isn't).  Introducing notions that add to the confusion
> > just to save a few cpu cycles and/or to make things look snappier
> > on-the-surface is no way to achieve that goal; unless you want a buggy,
> > insecure system but that niche is already well filled.
> 
> Or unless you want to have widespread use.  Unfortunately, the average user
> expects a computer to be fast.  By coupling the file extension method with
> a type-match check method when the file is opened using nautilus, you get
> the best of both worlds.  I fail to see the objection.  It's just as fast
> as the way Windows does it, and it's just as secure as the current way
> that nautilus does it.  Contriving a conflict is counter-productive.

We do have widespread use. Not as widespread as, say, Windows, but
still, we have many a user. That user-base is only increasing, as well.
The average user does not expect their computer to be as fast as you
claim. The average user, uses Windows. Windows is by no means fast.
Using both methods of file type detection will gain us nothing. In fact,
we already do use both methods. The suffix checking is generally a
fall-back method, though. Read the code. It's right there in plain
sight. Contriving conflicts is indeed counter-productive. Please stop.
:)

> > I wish this pointless discussion would go away.  It's clogging up my
> > inbox.  Really, there's some damn clever guys hacking Gnome and this
> 
> You can always unsubscribe to the list or request to get messages in a
> digest.  However, terminating a discussion because it bores you or is a
> minor inconvenience to you is not the right approach.
> [...snip...]

It's not only an annoyance to him. These mails have been going to far
more lists than they should have, if any at all. Continuing a thread
simply so that it does go on, is also not the right approach.

-- dobey