In VB6, what is the likely cause of an Overflow error? I am using strings when it occurs.
Is it RAM, or hard-drive space? Or is it something internal to VB6?
I am going to take a stab in the dark here, some code would help as Hogan said. Typically overflow error occur with string when VB6 things it is dealing with integer or longs in math formulas and the results are too large for a integer or long to hold.
Depending on the nature of your formula you may get away from the problem by forcing the system to use the numbers as floating point by adding a '.0" at the end. Otherwise use the various Cxxx function to explicitly convert the numbers to a type that has a greater range.
The one thing you consider is that floating point are less precise than whole number integers so make sure you don't lose needed precision when you do the conversion.
Another 'stab', since no code is provided ...
The following will produce an overflow error:
Dim x as integer
x = len(longstring) 'longstring over 32,768 characters in length
Would cause an overflow error.
Dim x as long
x = len(longstring) 'longstring over 32,768 characters in length
Works fine.
Another example of an overflow from Microsoft Support here:
EDIT
A more subtle situation that can catch you off guard:
You attempt to use a number in a calculation, and that number is coerced into an integer, but the result is larger than an integer.
Dim x As Long
x = 2000 * 365 ' Error: Overflow
To work around this situation, type the number, like this:
Dim x As Long
x = CLng(2000) * 365
Related
Everyone knows about overflow in the programming languages, if it happens program goes to crash. However, it is not clear for me what happens actually with data which get out of the boundary. Could you explain me, saying, giving example on C++ or Java. For example, Integer can save maximum 4 byte, what will happen if one puts data more than 4 byte to Integer. How compiler will identify this undefined behaviour?
what will happen if one puts data more than 4 byte to Integer.
Typically the value will roll-over1, meaning it will jump from one end of its range to another.
This can be seen, even in Windows calculator. Start with the highest possible signed 32-bit value:
Now add one to it:
We overflowed the maximum value of a signed Dword (231-1).
1 - This is a typical result. Some architectures might actually generate an exception on integer overflow, so you shouldn't count on this behavior.
How compiler will identify this undefined behaviour?
The compiler won't identify it. That's the problem. C# can mitigate this with the checked keyword, which checks to make sure that any arithmetic done on an integer will not cause overflow/underflow.
I'm using VBScript in ASP on IIS and I cannot seem to get the annoying error to go anyway so this makes debugging so much harder not to mention it does not tell me the exact error except the same error message so I can only assume what's wrong in my code.
My question is: Using the logical operators AND,XOR,NOT,OR in VBScript , is there a limit on the range to what the operands can be? I have implemented a bit shift right function and used the mod operator and I didn't notice until now that my function was causing the error.
My right shift function
function rshift(number,n)
'Shifts a number's bits n bits to the right
for i=1 to n
if number mod 2 = 0 then
number = number / 2
else
number = (number - 1) / 2
end if
next
rshift = number
end function
'Fails
rshift(1125899906842624,2)
I think for values larger than 2^32 ( or 31) - 1 that the operators do not work. Tried googling for the range of the operands but couldn't find anything helpful. I saw someone posted a topic about logical operators not working on large values but I can't seem to find that anymore.
Can someone verify this ?
Edit: Found a topic which gives more information on using the mod operator on signed 32 bit integers http://blogs.msdn.com/b/ericlippert/archive/2004/12/01/integer-arithmetic-in-vbscript-part-one.aspx
In VBScript there are several subtypes for integers including integer and long. VBScript will attempt to determine what type of value you are using and use the appropriate subtype.
Integer can store a value between -32,768 to 32,767 and long can store a value between -2,147,483,648 to 2,147,483,647. That value that you are using is greater than this and will result in an overflow.
You can use the VarType function to see what type your number is interpreted as. You may use Double to represent larger values.
This answer looks interesting. Maybe you can use it as part of your function.
double r = 11.631;
double theta = 21.4;
In the debugger, these are shown as 11.631000000000000 and 21.399999618530273.
How can I avoid this?
These accuracy problems are due to the internal representation of floating point numbers and there's not much you can do to avoid it.
By the way, printing these values at run-time often still leads to the correct results, at least using modern C++ compilers. For most operations, this isn't much of an issue.
I liked Joel's explanation, which deals with a similar binary floating point precision issue in Excel 2007:
See how there's a lot of 0110 0110 0110 there at the end? That's because 0.1 has no exact representation in binary... it's a repeating binary number. It's sort of like how 1/3 has no representation in decimal. 1/3 is 0.33333333 and you have to keep writing 3's forever. If you lose patience, you get something inexact.
So you can imagine how, in decimal, if you tried to do 3*1/3, and you didn't have time to write 3's forever, the result you would get would be 0.99999999, not 1, and people would get angry with you for being wrong.
If you have a value like:
double theta = 21.4;
And you want to do:
if (theta == 21.4)
{
}
You have to be a bit clever, you will need to check if the value of theta is really close to 21.4, but not necessarily that value.
if (fabs(theta - 21.4) <= 1e-6)
{
}
This is partly platform-specific - and we don't know what platform you're using.
It's also partly a case of knowing what you actually want to see. The debugger is showing you - to some extent, anyway - the precise value stored in your variable. In my article on binary floating point numbers in .NET, there's a C# class which lets you see the absolutely exact number stored in a double. The online version isn't working at the moment - I'll try to put one up on another site.
Given that the debugger sees the "actual" value, it's got to make a judgement call about what to display - it could show you the value rounded to a few decimal places, or a more precise value. Some debuggers do a better job than others at reading developers' minds, but it's a fundamental problem with binary floating point numbers.
Use the fixed-point decimal type if you want stability at the limits of precision. There are overheads, and you must explicitly cast if you wish to convert to floating point. If you do convert to floating point you will reintroduce the instabilities that seem to bother you.
Alternately you can get over it and learn to work with the limited precision of floating point arithmetic. For example you can use rounding to make values converge, or you can use epsilon comparisons to describe a tolerance. "Epsilon" is a constant you set up that defines a tolerance. For example, you may choose to regard two values as being equal if they are within 0.0001 of each other.
It occurs to me that you could use operator overloading to make epsilon comparisons transparent. That would be very cool.
For mantissa-exponent representations EPSILON must be computed to remain within the representable precision. For a number N, Epsilon = N / 10E+14
System.Double.Epsilon is the smallest representable positive value for the Double type. It is too small for our purpose. Read Microsoft's advice on equality testing
I've come across this before (on my blog) - I think the surprise tends to be that the 'irrational' numbers are different.
By 'irrational' here I'm just referring to the fact that they can't be accurately represented in this format. Real irrational numbers (like π - pi) can't be accurately represented at all.
Most people are familiar with 1/3 not working in decimal: 0.3333333333333...
The odd thing is that 1.1 doesn't work in floats. People expect decimal values to work in floating point numbers because of how they think of them:
1.1 is 11 x 10^-1
When actually they're in base-2
1.1 is 154811237190861 x 2^-47
You can't avoid it, you just have to get used to the fact that some floats are 'irrational', in the same way that 1/3 is.
One way you can avoid this is to use a library that uses an alternative method of representing decimal numbers, such as BCD
If you are using Java and you need accuracy, use the BigDecimal class for floating point calculations. It is slower but safer.
Seems to me that 21.399999618530273 is the single precision (float) representation of 21.4. Looks like the debugger is casting down from double to float somewhere.
You cant avoid this as you're using floating point numbers with fixed quantity of bytes. There's simply no isomorphism possible between real numbers and its limited notation.
But most of the time you can simply ignore it. 21.4==21.4 would still be true because it is still the same numbers with the same error. But 21.4f==21.4 may not be true because the error for float and double are different.
If you need fixed precision, perhaps you should try fixed point numbers. Or even integers. I for example often use int(1000*x) for passing to debug pager.
Dangers of computer arithmetic
If it bothers you, you can customize the way some values are displayed during debug. Use it with care :-)
Enhancing Debugging with the Debugger Display Attributes
Refer to General Decimal Arithmetic
Also take note when comparing floats, see this answer for more information.
According to the javadoc
"If at least one of the operands to a numerical operator is of type double, then the
operation is carried out using 64-bit floating-point arithmetic, and the result of the
numerical operator is a value of type double. If the other operand is not a double, it is
first widened (§5.1.5) to type double by numeric promotion (§5.6)."
Here is the Source
As I tried to debug, I found that : just as I type in
Dim value As Double
value = 0.90000
then hit enter, and it automatically converts to 0.9
Shouldn't it keep the precision in double in visual basic?
For my calculation, I absolutely need to show the precision
If precision is required then the Currency data type is what you want to use.
There are at least two representations of your value in play. One is the value you see on the screen -- a string -- and one is the internal representation -- a binary value. In dealing with fractional values, the two are often not equivalent and where they aren't, it's because they can't be.
If you stick with doubles, VB will maintain 53 bits of mantissa throughout your calculations, no matter how they might appear when printed. If you transition through the string domain, say by saving to a file or DB and later retrieving, it often has to leave some of that precision behind. It's inevitable, because the interface between the two domains is not perfect. Some values that can be exactly represented as strings (or Decimals, that is, powers of ten) can't be exactly represented as fractional powers of 2.
This has nothing to do with VB, it's the nature of floating point. The best you can do is control where the rounding occurs. For this purpose your friend is the Format function, which controls how a value appears in string form.
? Format$(0.9, "0.00000") will show you an example.
You are getting what you see on the screen confused with what bits are being set in the Double to make that number.
VB is simply being "helpful", and simply knocking off excess zeros. But for all intents and purposes,
0.9
is identical to
0.90000
If you don't believe me, try doing this comparison:
Debug.Print CDbl("0.9") = CDbl("0.90000")
As has already been said, displayed precision can be shown using the Format$() function, e.g.
Debug.Print Format$(0.9, "0.00000")
No, it shouldn't keep the precision. Binary floating point values don't retain this information... and it would be somewhat odd to do so, given that you're expressing the value in one base even though it's being represented in another.
I don't know whether VB6 has a decimal floating point type, but that's probably what you want - or a fixed point decimal type, perhaps. Certainly in .NET, System.Decimal has retained extra 0s from .NET 1.1 onwards. If this doesn't help you, you could think about remembering two integers - e.g. "90000" and "100000" in this case, so that the value you're representing is one integer divided by another, with the associated level of precision.
EDIT: I thought that Currency may be what you want, but according to this article, that's fixed at 4 decimal places, and you're trying to retain 5. You could potentially just multiply by 10, if you always want 5 decimal places - but it's an awkward thing to remember to do everywhere... and you'd have to work out how to format it appropriately. It would also always be 4 decimal places, I suspect, even if you'd specified fewer - so if you want "0.300" to be different to "0.3000" then Currency may not be appropriate. I'm entirely basing this on articles online though...
You can also enter the value as 0.9# instead. This helps avoid implicit coercion within an expression that may truncate the precision you expect. In most cases the compiler won't require this hint though because floating point literals default to Double (indeed, the IDE typically deletes the # symbol unless the value was an integer, e.g. 9#).
Contrast the results of these:
MsgBox TypeName(0.9)
MsgBox TypeName(0.9!)
MsgBox TypeName(0.9#)
Let's say I have any array Trial() As Integer
I have two following variables defined as Integer:
Dim Left As Integer
Dim Right As Integer
Now I am increasing the array index of trial
ReDim Preserve Trial(Left+Right)
Now If My total (Left+Right) exceeds Integer limit the above will give error.
And if redeclared Left as Long then it will work fine.
Actually I want to understand the internal calculation for (Left+Right).
Does it allocate the space for total depending on the datatype of "Left"?
Or it may also depends on datatype of "Right"?
It depends on both. The compiler will examine both variables and determine from both what data type it needs. For example. If you were to add (or multiple or divide) an Integer and a Long then the result will give you a long.
The calculation Left + Right is done presuming the result was an integer too, that's where the overflow occurs.
If you go CLng(Left) + CLng(Right) first, it is done the same way, only that the result will be a long, and thus no overflow occurs.