In a recent thread, I wanted to talk about the capacity of Double variables to store positive integers (in comparison with Long ones), and got a bit confused by what was written in the documentation.

For Long variables, the situation seems straightforward. As I understand it, a positive integer is then effectively stored in 63 bits, so that the largest positive integer which can be stored is 2^63, namely 9,223,372,036,854,775,808, and this corresponds exactly with what the documentation says …

In the case of a Double variable, my understanding is that a positive integer will be effectively (‘approximately’) stored in 52 bits of the mantissa/significant part of the storage, such that the largest positive integer which I would expect to be able to stored precisely would be about 2^52, namely 4,503,599,627,370,496 (i.e. 16 significant figures, up to the ceiling). However, the documentation says (of Double variables) …Originally Posted bythe documentation

This seems to be implying that the mantissa/significant can store 18 significant figures, two more than I would have expected.Originally Posted bythe documentation

Can someone help me to understand this apparent discrepancy/anomaly?

In an attempt to investigate this matter, interestingly, if I assign a positive integer value to a Double variable which has 18 significant figures, such as 912345678901234567, then if I display the value of that variable in, say, a textbox, I see just 15 significant figures (“9.12345678901235E+17”), the least significant one having been ‘rounded up’, and if I format it to show 18 significant figures, I still just get the same 15, followed by 3 trailing zeros (“912345678901235000”). However, if I use CLng() to convert the value to Long, and then display that value in a textbox, I see all 18 figures of the value which what was originally assigned to the Double variable, albeit the least significant figure is incorrect in terms of what was assigned (“912345678901234560”).

Whilst the precision I’m getting (for positive integers) with Double is roughly what I would have expected, can anyone help me to understand the above, and the documentation?

Just for the record, this code …

… produced the following in the three textboxes…Code:Dim x As Double = 912345678901234567 Dim y As Long = CLng(x) txt1.Text = CStr(x) txt2.Text = Format(x, "##################") txt3.Text = CStr(y)

txt1: 9.12345678901235E+17

txt1: 912345678901235000

txt1: 912345678901234560

Kind Regards, John