Nobody wanted to know this???

[SonGouki]

As it seems to be such a popular topic I've decided to attempt to teach all those who are wondering how numbers are stored in memory. Before I get started you should have at least rudimentary knowledge of how the different number systems (decimal, binary, and hex) work and how to convert between them, because I'm not going to explain that too.

Anyway here goes:
I'll start off with the more advanced stuff for those who already know the basics and just want to know the format.


IEEE Standard 754
This is the standard for floating point numbers in the current Intel architecture. This standard defines two formats: a single-precision floating point format (32-bits) and a double-precision floating point format (64-bits).

Single-Precision
Consists of 32-bits, which can be broken down as follows:

• 1 bit for the Sign of the Mantissa
• 8 bits for the Exponent (excess-127 notation)
• 23 bits for the normalized Mantissa (the leading bit is not stored because all normalized numbers start with 1, esentially meaning the Mantissa can provide 24 bits of precision)

The exponent range is from 2^-126 to 2^+127. This is because the stored exponent values 0 and 255 are used to indicate special values.

0.0 is stored with the exponent set to zero and the mantissa set to zero.

Double-Precision
Consists of 64-bits, which can be broken down as follows:

• 1 bit for the Sign of the Mantissa
• 11 bits for the Exponent (excess-1023 notation)
• 52 bits for the normalized Mantissa (same as above)

General:
So to lay out the bit format another way for both it looks generally like this:

SEE<->EEMMM<->MMM

S = Sign
E = Exponent
M = Mantissa (stored with the most significant bit at the left and the least significant bit at the right


Explanation
OK for all the rest of you out there who are probably thinking to yourselves "What the hell is this guy smoking?", I'll try to elaborate somewhat.
Note: All of the examples below will use the IEEE single-precision format.

The Floating Point Format
Can be broken down in to these parts from this example:
+0.258190 x 10^-6

The Mantissa "0.258190"
The Sign of the Mantissa "+"
The location of the Decimal Point
The number format of the Base "x 10"
The Exponent of the Base "6"
The Sign of the Exponent "-"

The Sign of the Mantissa
This is stored in the first bit, 0 represents a "Positive" mantissa, 1 represents a "Negative" mantissa.

The excess-N Exponent Format
For the bits allocated for the exponent, for example 8 bits in the IEEE single-precision format, a value is picked somewhere near the middle of the possible values for the exponent. In IEEE single this would be between 0 to 255, so a value of 127 (or 0111 1111 in Binary) is chosen as the base. Therefore, this means that an exponent of zero is represented as a value of 127 in the exponent bits, if an exponent is positive then its value is added onto the 127 base value, and negative is taken away. This is why it is known as excess-127 (or whatever the base value is).
For example:
an exponent of +5 would be stored as 132 (127 + 5 = 132)
an exponent of -10 would be stored as 117 (127 - 10 = 117)

Nomaralized Mantissa
The mantissa is the actual number that is being represented. In order to optimize the way that it is stored it must be normalized. To normalize a number the decimal place is shifted as far left as possible, by adjusting the exponent, until there are no leading zeros. The zero (to the left of the decimal place) is not stored in the mantissa because it is implied, thus the reason why it's called normalized.
Note: The normalized number can only be as long as the number of bits assigned to the mantissa (23 bits in IEEE single), if it is longer than this it is truncated (or rounded, I can't remmeber), if it is shorter then it is padded with zeros to fill up the mantissa bits
For example:
657.9107 normalized => 0.6579107 x 10^3
0.0007232 normalized => 0.7232 x 10^-4

Adjusting the Decimal Place
Just for those that don't know how to do it. In order to normalize the mantissa you must know how to adjust the decimal place of the number. When moving the decimal place left you increase the exponent, when moving the decimal place right you decrease the exponent.
For example:
7789.5 x 10^0
In order to normalize this number the decimal place would have to be moved 4 places to the left, thereby increasing the exponent +4.
= 0.77895 x 10^4

0.0000092 x 10^3
In order to normalize this number the decimal place would have to be moved 5 places to the right, thereby decreasing the exponent -5.
= 0.92 x 10^-2

Steps in Formatting the Number to Floating Point

• If there is no exponent for the number, then provide an exponent of zero.
• Shift the decimal point left or right until there are no leading zeros, thus normalizing the mantissa.
• Adjust the percision of the mantissa until it meets the specification of the floating point standard. Do this by either truncating the least significant digits (if too large) or padding the number with zeros (if not large enough) until it meets the required number of digits (such as 23 for IEEE single).
• Convert the number to the floating point storage format.

Note: I'm not going to show all the steps yet, not until I explain one more thing.
For example:
eg. 83574
Step 1 = 83574x10^0
Step 2 = 0.83574x10^5

Conversion to Binary (Base 2)
Now that you know the conversion method you also have to know that the system doesn't work with numbers in the decimal number system (Base 10). In order to see the way the computer really stores the numbers we have to convert all of the values to Binary notation (Base 2).I am not going to cover that here. There are plenty of places where you can learn how to do this.
What I will tell you is to keep in mind that when we convert to Binary the base changes. This means where before we had a base of 10 and then the exponent, now we have a base of 2 and the exponent. From here you can do either one of the following:
-Remove the exponent by adjusting the decimal place left or right. Then you convert the integer and fractional numbers to binary.
*OR*
-Remove the fractional part of the number by adjusting the exponent until there is only an integer number (ie. nothing after the decimal place). Now you convert the integer number to its Binary equivalent and into IEEE form, next you expand the Base and its exponent to its full form (eg. 10^3 would equal 1000). After that you convert the expanded exponent to its Binary equivalent and into IEEE form. Okay so now you should have two IEEE numbers, with these you have to divide the converted integer IEEE number by the converted exponent IEEE number using floating point division. Now I know you're asking "How do I perform floating point division?". Boy, you sure are demanding aren't you? OK, to do this you divide the two mantissas, subtract the excess of the two exponents, and then modify the sign of the mantissa in the normal way (ie. pos and neg = neg, neg and neg = pos, pos and pos = pos). Also, after this the number will need to be normalized again.

Note: When normalizing a Binary mantissa the advantage is that the numbers can only be either 0 or 1, therefore instead of normalizing the number to a 0 to the left of the decimal place we normalize the number so that the leftmost 1 is adjusted to the left of the decimal place. It is then dropped when it is converted to IEEE, and is always implied for the mantissa.

Okay, now for the whole process:
eg. -7830.28x10^3
First we remove the exponent.
= -7830280 (10)
Convert to Binary.
= -11101110111101100001000 (2)
Normalize the Mantissa.
= -1.1101110111101100001000x2^22 (2)
Write in IEEE 754 Single-Precision format.
= 1 | 10010101 | 11011101111011000010000

eg. 276718.75x10^-4
= 27.671875 (10)
= 11011.101011 (2)
= 1.1011101011x2^4 (2)
= 0 | 10000011 | 10111010110000000000000

Final Note
(and you thought I was done )
Okay so now that you know how to convert no problem (right!?!!), what you need to know is how the numbers are actually stored in the computer. As some you may know, numbers are stored in memory in reverse-byte order. Okay if don't know what this means here's an example (I don't feel like explaining it at this point!) taken from the first IEEE result from above.
Example:
IEEE example taken from above.
= 11001010111011101111011000010000
Convert in to Hex.
= 1100 1010 1110 1110 1111 0110 0001 0000
= CA EE F6 10
Which is represented in memory in reverse-byte format.
= 10 F6 EE CA


Whew! That's it! I hope this helps to straighten things out for everyone interested (and hopefully this didn't just confuse you more!). I apologize for any typos I made or ambiguities. I'll try to elaborate if somebody posts a request.

Later everybody!

BTW, post a response if this is what you guys were looking for, just so I can know.

[Eyes.Only]

very nice...wish i would have seen this a long time ago heh.

[techyspecy]

Great Stuff.

but boy you have got some patience and enthu.
How long it took you to write this stuff.
It took me half an hour to read and understand this stuff.

[moinkhan]

That's good stuff.. really useful...

[phinds]

(1) the following statement:

numbers are stored in memory in reverse-byte order

is NOT universally true. There are "little endian" systems and "big endian" systems. In "little endian" systems (Intel processors for example), the "little" byte comes first and thus the number is stored "backwards" as we humans view it. Not all systems are little endian. Motorola CPUs (Apple computers) are big endian.

(2) The most significant ramification of the conversion process which you so nicely describe, and which causes much headshaking in people who don't think it through, is the fact that there is an apparent weirdness in the results because binary numbers and decimal numbers do not have the same exact fractional values. For example, ".3" is a proper fracton in decimal, but it is NOT a proper fraction when converted to binary because it has to be represented as the sum of binary fractions that do not exactly add up to .3, so it gets saved exactly the same as if you had entered .29999999 and when converted back out, it comes out as .29999999 making some people think that there is something wrong with the process. It isn't "wrong" exactly, it's just an artifact of computers using binary instead of decimal.

(3) The example you give of normalization is correct but the way in which you describe the generic meaning of "normalize" is, I believe, not correct. This is just nit-picking and is not likely to be of any help to anyone, so I'm not going to bother will a full explaination.