Re: normalized strings in searches, completion, etc.

On 3/23/07, Denis Jacquerye <moyogo gmail com> wrote:
On 3/23/07, Sven Neumann <sven gimp org> wrote:
> Hi,
> On Fri, 2007-03-23 at 03:25 +0100, Denis Jacquerye wrote:
> > I'm sure there are tones of places where this doesn't work and some
> > where it does. But it should work everywhere someone does a search or
> > compares strings unless in some specific cases. What's the best way of
> > tackling the issue?
> It should work if all places where strings all compared would use
> g_utf8_collate(). I am surprised that this doesn't seem to be the case.
> Perhaps it's an issue that is often overlooked as many developers are
> not aware of the pitfalls of working with Unicode texts.

g_utf8_collate() uses G_NORMALIZE_ALL_COMPOSE = G_NORMALIZE_NFKC so it
will find ² and 2 equivalent. Should that be the default for all

Which is better? Using g_utf8_collate() instead of strcmp() or a
combination of g_utf8_normalize() and then strcmp()?
If g_utf8_normalize() is used, which normalization should be used?

I'm now guessing it should be G_NORMALIZE_NFC =
G_NORMALIZE_DEFAULT_COMPOSE in most cases because this will match
canonically equivalent strings (eg. é and é equivalent) but not
compatibility ones (eg. ² and 2 different). It will also not partially
match things like "Bise" with "Bisé" where the combining diacritic is
at the end of the string.

Actually, I take that back. Partial match would be inconsistent with
precomposed and those that can't be, eg. "bise" would not match "bisé"
but "bisɛ" would match "bisɛ́". So unless there's a better option,

I'm also guessing g_utf8_collate() is more appropriate for sorting
than for searching.

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]