hi @all, i'm new here, pls. excuse if i am not well adapted to the manners,
(asked something some time ago, got answer from a 'private email', but none to follow up questions, thus try to write to the list again, sorry if wrong,
>> 1. gnumeric produces much better sums over ranges than other spreadsheets which i have seen (LO calc and MS excel), can somebody explain by which means this is
>> achieved ('kahan summation' or 'summand ordering' or 'using FPU registers and long doubles' or other?) and / or give me a code pointer?
> 1. We use an extension of Kahan summation, see
https://gitlab.gnome.org/GNOME/goffice/-/blob/master/goffice/math/go-accumulator.c
thank you, i'm not a coder, looks nice and short and - as i have seen - works very good!,
is it ok to suggest the people at LibreOffice to have a look at this, there is one who wants to implement Kahan (with moderate success so far), and another is a little paranoid and thinks LibreOffice would get into trouble just by looking at your code / that of gnumeric at all, see
https://bugs.documentfoundation.org/show_bug.cgi?id=137679#c42
>> 3. gnumeric produces quite accurate results (3.000000000000000444E-01 for '=0.1 + 0.2'), but has weaknesses when rounding that to e.g. 16 decimals digits, results in
>> 3.000000000000000999E-01 which is 'enhancing the fail'. ...
> 3. This is based on misunderstandings. Our calculations are based on
base 2.
there are other questionable rounding results, e.g. '=rounddown(0.249999999999999972244,16)' results in 0.250000000000000000000, while '=roundup(0.249999999999999972244,16)' produces 0.249999999999999888978 which - acc. my simple math knowledge - should be the other way around? (0.249999999999999972244 results from calculating 0.25 minus one ULP, i'd expect a 'roundDOWN' always to be less or equal to a 'roundUP', i suspect a systematic error, but might be wrong in my expectations as well, bear with me if the latter ... ),
reg.
b.
