|
-
Feb 26th, 2020, 06:36 AM
#1
Is the SQLite Real Datatype "aproximate"?
I've picked up a project that has an issue. It uses C# with a SQLiteCommand, SqLiteParameters etc. to write a set of floats (singles) to a SQLite database. Subsequently, it reads the values back but reads and exposes them as doubles. (I don't know why there's a discrepancy at this point).
What I'm seeing is that floats rounded to 2DPs in C# are coming back into the doubles with a load of extra high precision decimal place values. E.g. I save 5.20, what I get back is 5.1999998092651300. Or I save 4.90, I get back 4.9000000953674300.
Interestingly, I don't even need to wait until the value comes back into a double before I see the problem. I've been using a SQLite viewer to look at the data and it's had the extra DPs added.
I know SQL Server Reals and Floats are "approximate". I.e. they can get extra high precision DPs drifting like this. My hunch is that this is what's happening inside SQLite, too, but I can't find anything in the documentation to confirm either way.
Does anyone know for sure what SQLite's behaviour is on this?
The best argument against democracy is a five minute conversation with the average voter - Winston Churchill
Hadoop actually sounds more like the way they greet each other in Yorkshire - Inferrd
-
Feb 26th, 2020, 07:06 AM
#2
Re: Is the SQLite Real Datatype "aproximate"?
According to the documentation at https://www.sqlite.org/datatype3.html
REAL. The value is a floating point value, stored as an 8-byte IEEE floating point number.
That seems to be the same as Double in .Net, so would cause this kind of issue (because the conversion to Double effectively happens on the way in to the database).
-
Feb 26th, 2020, 07:18 AM
#3
Re: Is the SQLite Real Datatype "aproximate"?
This sounds like the same:
https://stackoverflow.com/questions/...ion-in-room-db
Have you tried typecasting to double before INSERT?
Last edited by Zvoni; Tomorrow at 31:69 PM.
----------------------------------------------------------------------------------------
One System to rule them all, One Code to find them,
One IDE to bring them all, and to the Framework bind them,
in the Land of Redmond, where the Windows lie
---------------------------------------------------------------------------------
People call me crazy because i'm jumping out of perfectly fine airplanes.
---------------------------------------------------------------------------------
Code is like a joke: If you have to explain it, it's bad
-
Feb 26th, 2020, 08:02 AM
#4
Re: Is the SQLite Real Datatype "aproximate"?
Yep, that's it. Looks like if I explicitly cast it to a double it stores the full "precise" value. I managed to talk to the original developer as well and it looks like maintaining them as doubles throughout should be OK
Thanks both.
The best argument against democracy is a five minute conversation with the average voter - Winston Churchill
Hadoop actually sounds more like the way they greet each other in Yorkshire - Inferrd
-
Feb 29th, 2020, 09:55 PM
#5
Re: Is the SQLite Real Datatype "aproximate"?
 Originally Posted by FunkyDexter
Yep, that's it. Looks like if I explicitly cast it to a double it stores the full "precise" value.
IMO SQLite (when casting from a float- aka Single-value to a Double internally) does nothing wrong,
because when I do it "outside of SQLite" (in my case in the VB6-IDE Immediate-Window):
?CDbl(5.20!) ' <- the ! enforcing a Single-type on the Num-Literal
... it Prints out:
5.19999980926514
So not really sure what you mean (or did), when you say you can avoid this with "explicit casting" (on the outside).
Olaf
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|