I'm getting odd results in some VB6 code which I've narrowed to this:
Debug.Print Hex(&hEDB80000 + &h8300)
Shows EDB78300
That can't by right can it? Surely it should be EDB88300?
Am I going mad?
Don't forget how negative numbers are expressed in binary, and that VB6 and VB.NET interpret numbers like &h8300 differently.
Because &hEDB80000 doesn't fit in 16-bits, VB interprets it as a long (32-bits). Because the high bit is set, VB6 knows it's negative.
Let's undo the two's complement (in a 32-bit world) to figure out the decimal value
(~&hEDB80000 + 1) = &h1247FFFF + 1 = &h12480000 = 306708480
since the sign bit was set, that's -306708480
Because &h8300 fits in 16-bits, VB interprets it as an integer (16-bits). Because the high bit is set, VB6 knows that it's negative.
Let's undo the two's complement (in a 16-bit world)
(~&h8300 + 1) = &h7DFF + 1 = &h7D00 = 32000
since the sign bit was set, that's -32000. When the addition happens, both values are considered to be longs (32-bits).
(-306708480) + (-32000) = -306740480
Let's put that back into two's complement hex
~(306740480 - 1) = ~(&h12487D00 - 1) = ~(&h12487CFF) = &hEDB78300
So &hEDB78300 is the correct answer.
Notes:
I personally thing the confusion happens because of the following:
&h0004000 is interpreted as 16384 // Fits in 16-bits, sign bit is not set
&h0008000 is interpreted as -32768 // Fits in 16-bits, sign bit is set
&h0010000 is interpreted as 65536 // Requires 32-bits, sign bit is not set
as mentioned in the other post, you can get around this by explicitly marking values as longs
&h0004000& is interpreted as 16384
&h0008000& is interpreted as 32768
&h0010000& is interpreted as 65536
Fundementally because VB6 sees &h8300 as an integer having the value -32000. To get the results you were expecting you would need to explictly mark it as a Long:-
Debug.Print Hex(&hEDB80000 + &h8300&)
What your were doing was adding a Long to an Interger. To do that VB6 first extends the Integer to a Long, since &h8300 represents a negative number the Long it is converted to ends up with the value &hFFFF8300. Armed with that value you can see that the result returned by VB6 is correct.
FF + B8 = B7 with carry bit set
FF + ED + carry bit = ED
Related
How do I print long values in gdb?
When using just x, i.e x $rdi, the value (in hex) is cut off after 8 bytes.
If I use x/32bx(or whatever other length), bytes are separated by spaces which is not nice, but okay. The problem is that when there's some long value I want to print, I have to guess the size and pass it to x/. If that value is 256 bytes long, the output will look messy, because it's separated by spaces, but it also means I have to make a lot of guesses and then look through a long and ugly string of bytes and find the place where the value ends and is followed by 0x00s (and obviously the value can have 0x00s in between which makes trying to work this out even more confusing) to be able to know how long it is.
If I try to print it as an integer, it gets cut off as well. I'd like to be able to easily tell how long a value is and not have it be cut off.
A way to display long (as well as ll & ull) values is with the g modifier. For example if we have a program which just stores
unsigned long long int a = 1234567891234567898;
int b = 23;
unsigned long long int c = 1111111111111111111;
by typing x/20xg $rsp (after the values have been moved to the stack) we get
0x7fffffffdd90: 0x0000000000000000 0x0000001755555040
0x7fffffffdda0: 0x112210f4c023b6da 0x0f6b75ab2bc471c7
0x7fffffffddb0: 0x0000000000000000 0x00007ffff7e08b25
...
With the long numbers in [rsp+0x10] & [rsp+0x18] being a & c respectively, and that 0x17 in [rsp+0xc] being b.
The documentation for the OUTPUT_DEBUG_STRING_INFO structure doesn't explain, how to determine the length (or size) of the string value it points to. Specifically, the documentation for nDebugStringLength is confusing:
The lower 16 bits of the length of the string in bytes. As nDebugStringLength is of type WORD, this does not always contain the full length of the string in bytes.
For example, if the original output string is longer than 65536 bytes, this field will contain a value that is less than the actual string length in bytes.
As I understand it, the true size can be any value that's a solution to the equation:
size = nDebugStringLength + (n * 65536)
for any n in [0..65536).
Question:
How do I determine the correct size of the string? Unless I'm overlooking something, the documentation appears to be insufficient in this regard.
initially the debug event comes in the form DBGUI_WAIT_STATE_CHANGE
if use WaitForDebugEvent[Ex] api - it internally convert DBGUI_WAIT_STATE_CHANGE to DEBUG_EVENT by using DbgUiConvertStateChangeStructure[Ex]
the DbgExceptionStateChang ( in NewState) event with DBG_PRINTEXCEPTION_WIDE_C and DBG_PRINTEXCEPTION_C (in ExceptionCode) converted to OUTPUT_DEBUG_STRING_INFO. the nDebugStringLength is taken from Exception.ExceptionRecord.ExceptionInformation[0] or from ExceptionInformation[3] (in case DBG_PRINTEXCEPTION_C and api version without Ex ). but because nDebugStringLength is only 16 bit length, when original value is 32/64 bit length - it truncated - only low 16 bit of ExceptionInformation[0] (or [3]) is used.
note that ExceptionInformation[0] (and [3] in case DBG_PRINTEXCEPTION_WIDE_C ) containing string length in characters, including terminating 0.
in contrast nDebugStringLength in bytes (if we using WaitForDebugEventEx and DBG_PRINTEXCEPTION_WIDE_C exception - nDebugStringLength = (WORD)(ExceptionInformation[0] * sizeof(WCHAR))
This is floating point computation issue (i think).
This is 2 round function i found somewhere.
They look very similar in code but when i run it, i get 2 different result.
In this example when run this code with value is 87.13225 and precision 4 :
a = 87.1322
b = 87.1323
Anyone can explain what happen?
Private Sub Form_Load()
Dim a#
Dim b#
a = Round1(87.13225, 4) '87.1322
b = Round2(87.13225, 4) '87.1323
End Sub
Private Function Round1(ByVal value#, ByVal vPrecision%)
Round1 = Fix((value + Sgn(value) / 10 ^ vPrecision / 2) * _
10 ^ vPrecision) _
/ 10 ^ vPrecision
End Function
Private Function Round2(ByVal value#, ByVal vPrecision%)
Dim a#
a = (value + Sgn(value) / 10 ^ vPrecision / 2) * 10 ^ vPrecision
Round2 = Fix(a) / 10 ^ vPrecision
End Function
Part of your problem, I believe, involves type conversion. Since the literal parameters you are passing are not defined as double, I believe they are being passed as singles, which then get converted to doubles, and this likely affects your outcomes. Try passing the literal to each method as 87.12335#. Also, you're not defining either function's output type, so I believe it's returning variants. I believe both methods should end with " As Double"
Finally, why are you attempting to re-invent the wheel? VB6 is capable of two difference kinds of rounding. Banker's Rounding is the default rounding scheme for VB6. This surprised me, and I write accounting software. Banker's rounding always rounds to the nearest EVEN number, so both of your results should be 87.1234. In banker's rounding, 87.12345 would ALSO round to 87.1234. If you want the rounding where it always rounds away from zero, which is what I was more familiar with, you should use the various format methods, i.e. FormatNumber, Format$, etc. There is a link on the page referenced above that explains this further. When I discovered this discrepancy, I made my own rounding routine with uses either Banker's rounding or what I call Standard rounding, based upon what my customer prefers.
I have a fairly complex look of code where I am looking through multiple control variables.
I am getting an error 'Invalid 'for' loop control variable
the line in questions is
for w(1) = 32 to 127
I am more familiar with VBA where I would have zero problem with this statement.
I'm guessing it has something to do with the fact that i will be looping through w(1),w(2),w(3) etc. in the same tree. I initialize the variable as dim x(10) but have also tried dim w() , dim w() redim w(10)
Any thoughts? its a fairly critical aspect of the script; as such I am unwilling to swap out all my w 1,2... for individual variables
Thoughts?
EDIT:
As per comments I should clarify a Few things:
Essentially there is a alpha numeric association with an ID in a system that I am working with which I was not handed down the key too. So I have a multi-dimensional array of rates that are used for multiplying out costs.
What I am doing is working backwards through invoices and matching a material with very subtle differences that have different pricings.
For simplicity sake, say theres a 2 dimensional material where AA, AB, ... A9 are all priced through several multiplication factors in what would just be a 2x2 grid. So maintaining a pivot point based on the position in string is very important. For this code you could take tier to mean how many characters in the string (aka how complex the composition of the material):
dim x(), w()
for tier = 1 to 2
for w(1) = 32 to 127
x(1)= chr(w(1))
If tier = 2 then
for w(2)= 32 to 127
X(2)=chr(w(2))
next
end if
str = ""
for y = 1 to (tier)
str = trim(str & x(y))
next
'''msgbox str 'debug
next
end if
str = ""
for y = 1 to (tier)
str = trim(str & x(y))
next
'' msgbox str ' debug
next 'tier
This is just an excerpt i pulled to get a basic idea of the structure w/o any calculations. this is in essence what is not working
The error is quite clear, you cannot use an Array as the control variable. The definition in For...Next Statement is even clearer;
Numeric variable used as a loop counter. The variable cannot be an array element or an element of a user-defined type.
This is one of the key differences between VBA and VBScript.
You won't loop through x(1),x(2)...on what you write it's going like this 32(1),33(1)....what type it's your w(1) and how you define him?
When we convert a float to integer in visual basic 6.0, how does it round off the fractional part? I am talkin about the automatic type conversion.
If we assign like
Dim i as Integer
i=5.5
msgbox i
What will it print? 5 or 6 ??
I was getting "5" a couple of months before. One day it started giving me 6!
Any idea whats goin wrong? Did microsoft released some patches to fix something?
Update : 5.5 gets converted to 6 but 8.5 to 8 !
Update 2 : Adding CInt makes no difference. CInt(5.5) gives 6 and Cint(8.5) gives 8!! Kinda weired behaviour. I should try something like floor(x + 0.49);
Part of this is in the VB6 help: topic Type Conversion Functions. Unfortunately it's one of the topics that's not in the VB6 documentation on the MSDN website, but if you've installed the help with VB6, it will be there.
When the fractional part is exactly 0.5, CInt and CLng always round it to the nearest even number. For example, 0.5 rounds to 0, and 1.5 rounds to 2. CInt and CLng differ from the Fix and Int functions, which truncate, rather than round, the fractional part of a number. Also, Fix and Int always return a value of the same type as is passed in.
Implicit type coercion - a.k.a. "evil type coercion (PDF)" - from a floating point number to a whole number uses the same rounding rules as CInt and CLng. This behaviour doesn't seem to be documented anywhere in the manual.
If you want to round up when the fractional part is >= 0.5, and down otherwise, a simple way to do it is
n = Int(x + 0.5)
And off the top of my head, here's my briefer version of Mike Spross's function which is a replacement for the VB6 Round function.
'Function corrected, now it works.
Public Function RoundNumber(ByVal value As Currency, Optional PlacesAfterDecimal As Integer = 0) As Currency
Dim nMultiplier As Long
nMultiplier = 10 ^ PlacesAfterDecimal
RoundNumber = Fix(0.5 * Sgn(value) + value * nMultiplier) / CDbl(nMultiplier)
End Function
Sample output:
Debug.Print RoundNumber(1.6) '2'
Debug.Print RoundNumber(-4.8) '-5'
Debug.Print RoundNumber(101.7) '102'
Debug.Print RoundNumber(12.535, 2) '12.54'
Update: After some googling, I came across the following article:
It is not a "bug", it is the way VB was
designed to work. It uses something
known as Banker's rounding which, if
the number ends in exactly 5 and you
want to round to the position in front
of the 5, it rounds numbers down if
the number in front of the 5's
position is even and rounds up
otherwise. It is supposed to protect
against repeated calculation using
rounded numbers so that answer aren't
always biased upward. For more on this
issue than you probably want to know,
see this link
http://support.microsoft.com/default.aspx?scid=KB;EN-US;Q196652
This explains the (apparent) weird behavior:
Cint(5.5) 'Should be 6'
Cint(8.5) 'Should be 8'
Old Update:
Perhaps you should be more explicit: use CInt, instead of simply assigning a float to an integer. E.g:
Dim i as Integer
i = CInt(5.5)
MsgBox i
The changed behaviour sounds worrying indeed, however the correct answer surley is 6. Scroll down to "Round to even method" on Wikipedia, Rounding for an explanation.
As others have already pointed out, the "weird behavior" you're seeing is due to the fact that VB6 uses Banker's Rounding when rounding fractional values.
Update 2 : Adding CInt makes no
difference. CInt(5.5) gives 6 and
Cint(8.5) gives 8!!
That is also normal. CInt always rounds (again using the Banker's Rounding method) before performing a conversion.
If you have a number with a fractional part and simply want to truncate it (ignore the portion after the decimal point), you can use either the Fix or the Int function:
Fix(1.5) = 1
Fix(300.4) = 300
Fix(-12.394) = -12
Int works the same way as Fix, except for the fact that it rounds negative numbers down to the next-lowest negative number:
Int(1.5) = 1
Int(300.4) = 300
Int(-12.394) = -13
If you actually want to round a number according to the rules most people are familiar with, you will have to write your own function to do it. Below is an example rounding that will round up when the fractional part is greater than or equal to .5, and round down otherwise:
EDIT: See MarkJ's answer for a much simpler (and probably faster) version of this function.
' Rounds value to the specified number of places'
' Probably could be optimized. I just wrote it off the top of my head,'
' but it seems to work.'
Public Function RoundNumber(ByVal value As Double, Optional PlacesAfterDecimal As Integer = 0) As Double
Dim expandedValue As Double
Dim returnValue As Double
Dim bRoundUp As Boolean
expandedValue = value
expandedValue = expandedValue * 10 ^ (PlacesAfterDecimal + 1)
expandedValue = Fix(expandedValue)
bRoundUp = (Abs(expandedValue) Mod 10) >= 5
If bRoundUp Then
expandedValue = (Fix(expandedValue / 10) + Sgn(value)) * 10
Else
expandedValue = Fix(expandedValue / 10) * 10
End If
returnValue = expandedValue / 10 ^ (PlacesAfterDecimal + 1)
RoundNumber = returnValue
End Function
Examples
Debug.Print RoundNumber(1.6) '2'
Debug.Print RoundNumber(-4.8) '-5'
Debug.Print RoundNumber(101.7) '102'
Debug.Print RoundNumber(12.535, 2) '12.54'
The VB6 Round() function uses a Banker's Rounding method. MS KB Article 225330 (http://support.microsoft.com/kb/225330) talks about this indirectly by comparing VBA in Office 2000 to Excel's native behavior and describes it this way:
When a number with an even integer ends in .5, Visual Basic rounds the number (down) to the nearest even whole number. [...] This difference [between VBA and Excel] is only for numbers ending in a .5 and is the same with other fractional numbers.
If you need different behavior, I'm afraid you'll have to have to specify it yourself.