|
-
Aug 3rd, 2019, 12:11 PM
#4
Re: Overflow error
Hi tmighty2,
VB6 has the nice Decimal type that can be used for these very large integers. You can only get to them through the use of a Variant, and you can only assign to them using the CDec() function. But, once you've got your variables (Decimals in Variants), you can do all the typical math with them, and they'll stay Decimals, not overflowing until you hit MUCH larger numbers than yours. They're effectively a 12 byte unsigned integer with a separate sign bit.
Here are some of the usage rules for them:
Code:
' +, -, *, / Decimal is at the top of the hierarchy.
' The hierarchy is: Byte, Integer, Long, Single, Double, Currency, Decimal.
' \, Mod Results is Long, not Decimal.
' ^ Results is Double, not Decimal.
' Int() Works on Decimals.
' Fix() Works on Decimals.
' Abs() Works on Decimals.
' Sgn() Works on Decimals, although result is an Integer.
' Sqr(), etc. Most of these functions (including trig) return Double, not Decimal.
' AND,OR,etc. These convert to Long, which may cause overflow.
'
' Largest Decimal: +/- 79228162514264337593543950335. 2^96-1 (sign bit handled separately)
' Smallest Decimal: +/- 0.0000000000000000000000000001 Notice that both largest and smallest are same width.
Good Luck,
Elroy
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
Tags for this Thread
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|