Quantcast
Channel: Rainmeter Forums
Viewing all articles
Browse latest Browse all 993

Bugs & Feature Suggestions • [NOT A BUG] Floating Point Imprecision

$
0
0
This looks like general floating point shenanigans, and can even mess with comparisons so depending on what I'm doing I tend to round values before doing anything with them to deal with that extra .0000001 or whatever that may be in there.
Good idea, though in this case the rounding must be done in the proper stage (i.e. Formula=(Ceil(Abs(Log(Abs(Round(1-Frac(4294967295.999),5))))))), to not defeat the formula's purpose (i.e. to compute the number of integral and decimal digits from a number with as much precision as possible using only numerical functions).

Thanks for the idea, sometimes the easiest solutions are the easiest to miss. In the end, I used - what else - the never failing strings / regexes to do the above in my actual code, and ditched the numerical way entirely, but I'll keep your idea in mind the next time I'll need it (hopefully I'll also remember to not post yet another topic regarding these things here). :thumbup:

Statistics: Posted by Yincognito — Yesterday, 6:12 pm — Replies 4 — Views 576



Viewing all articles
Browse latest Browse all 993

Trending Articles