I want to know how many number of digits are allowed after decimal point for primitive double datatype in vb6, without actually getting rounded off.
You get up to 16 significant figures in case of double data type in vb6 with accuracy
eg 1.0000000000000006
Related
I am confused to see SAS Numeric Format BEST15.2 is allowing more than two palces after decimal. What is the correct interpretation of BEST15.2
Looking up the documentation the Best format only has a width specification, not a decimal specification.
Further, the documentation does say:
Numbers with decimals are written with as many digits to the left and
right of the decimal point as needed or as allowed by the width.
which might explain what you are seeing.
An alternative could be the BESTDw.p format which allows you to specify the decimal precision:
Prints numeric values, lining up decimal places for values of similar
magnitude, and prints integers without decimals.
This line:
NSLog(#"DBL_MAX: %f", DBL_MAX);
prints this very large value:
17976931348623157081452742373170435679807056752584499659891747680315726078002853876058955863276687817154045895351438246423432132688946418276846754670353751698604991057655128207624549009038932894407586850845513394230458323690322294816580855933212334827479
However, when I test a double value like this:
double test = 9999999999999999.0;
NSLog(#"test: %f", test);
I get this unexpected result:
test: 10000000000000000.000000
This appears to be the maximum number of digits that produce the expected result:
double test = 999999999999999.0;
NSLog(#"test: %f", test);
test: 999999999999999.000000
How can I work with higher positive fractions?
Thanks.
Unfortunately, I can't answer the question directly, as I don't understand what you mean by "How can I work with higher positive fractions?".
However, I can shed some light on what a floating-point number is ans what it isn't.
A floating-point number consists of:
A sign (plus or minus)
An exponent
A value (known as the "mantissa").
These are combined using a clever encoding typically into 32, 64, 80, or 128 bits. In addition, some special encodings are used to represent +-infinity, Not a Number (NaN), and +-Zero.
As the mantissa has a limited number of bits, your value can only have this number of significant bits. A really small floating-point number can represent values in the 10^-308 and and large one 10^308. However, any number can only have about 16 decimal digits.
On other words, the print-out if DBL_MAX does not corresponds the amount of information stored in the variable. For example, there is no way to represent the same number but with a ...7480 instead of ...7479 at the end.
So back to the question, in order to tell how to represent your values, you must describe what kind of values you want to represent. Are they really fractions (i.e. one integer divided by another integer), in that case you might want to represent this using two integers. If you want to represent really large values, you might want to use packages like http://gmplib.org
Floating point in C# doesn't produce accurate results all the time. There are numbers that cannot be represented in double, floats or decimals. You can improve your accuracy by using "decimal" instead of "double", but it still doesn't ensure that all numbers will be represented exactly.
I'm using the conversion function CStr in VBScript to convert a number of type Long into a String. The problem is after converting to a string the decimal point is lost eg 2.5 is converted to 2.
Does anyone know a way to preserve the decimal point using this conversion function?
Many Thanks,
Its probably that casting the number to Long is the problem as long is just a type that will contains bigger integers, and does not support decimals.
Maybe try a currency, single or double type for the numeric value before doing the CStr
Have you echoed out your CLng variable? CLng rounds decimals to the nearest even number so it is likely the CLng which is rounding your decimal places rather than CStr dropping them. Can you use CDbl rather than CLng?
As I tried to debug, I found that : just as I type in
Dim value As Double
value = 0.90000
then hit enter, and it automatically converts to 0.9
Shouldn't it keep the precision in double in visual basic?
For my calculation, I absolutely need to show the precision
If precision is required then the Currency data type is what you want to use.
There are at least two representations of your value in play. One is the value you see on the screen -- a string -- and one is the internal representation -- a binary value. In dealing with fractional values, the two are often not equivalent and where they aren't, it's because they can't be.
If you stick with doubles, VB will maintain 53 bits of mantissa throughout your calculations, no matter how they might appear when printed. If you transition through the string domain, say by saving to a file or DB and later retrieving, it often has to leave some of that precision behind. It's inevitable, because the interface between the two domains is not perfect. Some values that can be exactly represented as strings (or Decimals, that is, powers of ten) can't be exactly represented as fractional powers of 2.
This has nothing to do with VB, it's the nature of floating point. The best you can do is control where the rounding occurs. For this purpose your friend is the Format function, which controls how a value appears in string form.
? Format$(0.9, "0.00000") will show you an example.
You are getting what you see on the screen confused with what bits are being set in the Double to make that number.
VB is simply being "helpful", and simply knocking off excess zeros. But for all intents and purposes,
0.9
is identical to
0.90000
If you don't believe me, try doing this comparison:
Debug.Print CDbl("0.9") = CDbl("0.90000")
As has already been said, displayed precision can be shown using the Format$() function, e.g.
Debug.Print Format$(0.9, "0.00000")
No, it shouldn't keep the precision. Binary floating point values don't retain this information... and it would be somewhat odd to do so, given that you're expressing the value in one base even though it's being represented in another.
I don't know whether VB6 has a decimal floating point type, but that's probably what you want - or a fixed point decimal type, perhaps. Certainly in .NET, System.Decimal has retained extra 0s from .NET 1.1 onwards. If this doesn't help you, you could think about remembering two integers - e.g. "90000" and "100000" in this case, so that the value you're representing is one integer divided by another, with the associated level of precision.
EDIT: I thought that Currency may be what you want, but according to this article, that's fixed at 4 decimal places, and you're trying to retain 5. You could potentially just multiply by 10, if you always want 5 decimal places - but it's an awkward thing to remember to do everywhere... and you'd have to work out how to format it appropriately. It would also always be 4 decimal places, I suspect, even if you'd specified fewer - so if you want "0.300" to be different to "0.3000" then Currency may not be appropriate. I'm entirely basing this on articles online though...
You can also enter the value as 0.9# instead. This helps avoid implicit coercion within an expression that may truncate the precision you expect. In most cases the compiler won't require this hint though because floating point literals default to Double (indeed, the IDE typically deletes the # symbol unless the value was an integer, e.g. 9#).
Contrast the results of these:
MsgBox TypeName(0.9)
MsgBox TypeName(0.9!)
MsgBox TypeName(0.9#)
I'm having an overflow error in VB 6.0 when using the Clong datatype because of really big values. How to overcome this? Is there anything else available higher than the Clong datatype?
Depending on how big your really big values are, the VB6 Currency data type might be a good choice.
It supports values in the range -922,337,203,685,477.5808 to 922,337,203,685,477.5807.
You could use a Double instead of a Long since it can hold larger numbers. The function is CDbl() instead CLng().
In VB6.0, a Long is 32-bits and can hold values up to: 2,147,483,648
A Double is 64-bits and can old values up to: 1.79769313486231570E+308
EDIT: Please refer to this reference
I believe the upcoming VB in MSVS2010 has the CLonger (64 bits), CEvenLongerYet (128 bits) and CTooDamnLongForSensibleUse (256 bits) data types.
</humor>
Here are some options from the VB6 reference manual topic on data types
Long (long integer) 4 bytes
-2,147,483,648 to 2,147,483,647
Single (single-precision
floating-point) 4 bytes -3.402823E38
to -1.401298E-45 for negative values;
1.401298E-45 to 3.402823E38 for positive values. About 6 or 7 significant figures accuracy.
Double
(double-precision floating-point) 8
bytes -1.79769313486231E308 to
-4.94065645841247E-324 for negative values; 4.94065645841247E-324 to
1.79769313486232E308 for positive values. About 15 or 16 significant figures accuracy.
Currency (scaled integer) 8
bytes -922,337,203,685,477.5808 to
922,337,203,685,477.5807
Decimal 14
bytes
+/-79,228,162,514,264,337,593,543,950,335
with no decimal point;
+/-7.9228162514264337593543950335 with 28 places to the right of the
decimal; smallest non-zero number is
+/-0.0000000000000000000000000001
Try avoiding division by zero. If the numerator and denominator object of your code is equal to zero, try making the denominator equal to 1. hence, zero/zero = overflow
zero/1 = zero ( no overflow)