Skip to content

Floating point loses precision #69

@dcleblanc

Description

@dcleblanc

An AI review of the code found an issue where some large floating point numbers will lose precision.

For example, a number that uses 54 bits will lose a bit when converted to double because a double only has 53 bits for the value. I feel like this is by design, you asked for a floating point. You get into the same sort of problem when converting from double to say uint64_t. The converted value will leave some number of zeros as the least significant bits.

Proposed fix:

Add (yet another) compile time flag that governs whether it will be strict about floating point conversions, and if the flag is set then turn these into exceptions.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions