Re: Text editing and UTF-8 conversion

[Tor Lillqvist <tml iki fi>]

> Is there anything that I could do differently to increase the loading speed
> or is this just something I need to just deal with and move on to the next
> item on the list?

You need to find out what exactly it is that is taking a lot of time,
and experiment with different ways to do that. I.e., write separate
test programs to explore different strategies for:

- conversion. Do you convert a line at a time, some buffer of some
length at a time, all the file at a time, or what?

- reading in the file data. Do you read the whole file into memory in
one go? or a line at a time? or what?

Finally, if you are serious that you want to support large files
(i.e., files larger than 2 GB), you need to take this very much into
consideration from the start when designing your data structures and
what features the program actually offers. You can't just read in a
multi-gigabyte file and expect to be able to handle that data like you
would a much smaller file. (In a 64-bit program on a 64-bit OS and on
a machine with enough RAM, you can, but it will be slow, and I don't
know if Visual C++ Express supports 64-bit code.)

--tml