Pass a pointer in a win32 dll from vbscript (qtp) - winapi

I have a good old fashioned win32 dll with functions of the form
void Foo1(int* value)
void Foo2(char* string)
void Foo3(MyType* data)
//ect...
I need to call this in QTP (vbscript) and retreive the data for use in the QTP application. Is this even possible in vbscript?
I have some controll over the DLL. It is written in c++. Building a COM server is not an option. Refactoring the code to include accessor methods with ordinal return types is flat out of the question (would be a maintainance and scabaility nightmare).
Editing to clarify the example...
I have...
void Add(int x, int y, int* result)
...I need to do the QTP equivalent of this...
int myX = 2;
int myY = 5;
int myResult = -1;
Add(myX, myY, &myResult);
//myResult should now be 7
...but in QTP.
Calling int Bar(int x, int y) in QTP is easy.
I need to know if its possible to call into void Foo(int* result)
in this way Foo(&myResult) and pass in a reference to result.

You can declare external functions like Win32 API but there are some limitations, I believe the function should have extern "C" linkage and not all types are supported. Here's an example of how to use the Win32 GetParent function from QTP, you may be able to extrapolate how to match your own function.
' Declare
Extern.Declare micHwnd, "GetParent", "user32.dll", "GetParent", micHwnd
' explanation: retVal name sourceDll name parameter
' Usage
hwnd = Extern.GetParent(Browser(“xxx”).GetROProperty("hwnd"))
The reason the name appears twice is so you can rename it for your script (I don't remember which is the name to look for and which is the name that you will use).

The answer is to pass by reference.
In the .h file declare the function:
extern "C" __declspec(dllexport) int HelloWorld(int &);
In the .cpp file define the function:
int HelloWorld(int& i)
{
i = 10;
return 55 + i;
}
QTP code:
Extern.Declare micInteger, "HelloWorld", "C:\test.dll", "HelloWorld", micInteger+micByRef
myNumber = 5
Extern.HelloWorld(myNumber)
MsgBox(myNumber)

Related

What is the "address" being returned from GetProcAddress()?

I'm a little bit confused with GetProcAddress().
Quoting the Win32 docs:
If the function succeeds, the return value is the address of the exported function or variable.
I know that for functions, GetProcAddress() returns a function pointer that calls the desired function. However, it isn't clear what GetProcAddress() returns in something like this:
DLL code
typedef struct {
uint64_t foo;
double bar;
char* baz;
} MyStruct;
__declspec(dllexport) MyStruct* my_struct_ptr;
__declspec(dllexport) long long special_global;
Application code
void* my_struct_ptr = GetProcAddress(my_dll_handle, "my_struct_ptr");
void* special_global = GetProcAddress(my_dll_handle, "special_global");
What would my_struct_ptr and special_global in the application point to?
EDIT: Does my_struct_ptr point to the DLL's my_struct_ptr or the struct that the DLL's my_struct_ptr points to?

duplicate symbol of a function defined in a header file

Suppose I have a header file file_ops.hpp that looks something like this
#pragma once
bool systemIsLittleEndian() {
uint16_t x = 0x0011;
uint8_t *half_x = (uint8_t *) &x;
if (*half_x == 0x11)
return true;
else
return false;
}
I initially thought it had something to do with the implementation, but as it turns out, I'll get duplicate symbols with just
#pragma once
bool systemIsLittleEndian() { return true; }
If I make it inline, the linker errors go away. That's not something I want to rely on, since inline is a request not a guarantee.
What causes this behavior? I'm not dealing with a scenario where I'm returning some kind of singleton.
There are other methods that are marked as
bool MY_LIB_EXPORT someFunc();// implemented in `file_ops.cpp`
are these related somehow (mixed exported functions and "plain old functions")? Clearly I can just move the implementation to file_ops.cpp, I'm rather intrigued as to why this happens.
If I make it inline, the linker errors go away. That's not something I want to rely on, since inline is a request not a guarantee.
It's OK to inline the function.
Even if the object code is not inlined, the language guarantees that is will not cause linker errors or undefined behavior as long as the function is somehow not altered in different translation units.
If you #include the .hpp in hundreds of .cpp files, you may notice a bit of code bloat but the program is still correct.
What causes this behavior? I'm not dealing with a scenario where I'm returning some kind of singleton.
The #include mechanism is a convenience for reducing the amount of code you have to manually create in multiple files with the exact content. In the end, all translation units that #include other files get the lines of code from the files they #include.
If you #include file_ops.hpp in, let's say, file1.cpp and file2.cpp, it's as if you have:
file1.cpp:
bool systemIsLittleEndian() {
uint16_t x = 0x0011;
uint8_t *half_x = (uint8_t *) &x;
if (*half_x == 0x11)
return true;
else
return false;
}
file2.cpp:
bool systemIsLittleEndian() {
uint16_t x = 0x0011;
uint8_t *half_x = (uint8_t *) &x;
if (*half_x == 0x11)
return true;
else
return false;
}
When you compile those two .cpp files and link them together to create an executable, the linker notices that there are two definitions of the function named systemIsLittleEndian. That's the source of the linker error.
One solution without using inline
One solution to your problem, without using inline, is:
Declare the function in the .hpp file.
Define it in the appropriate .cpp file..
file_ops.hpp:
bool systemIsLittleEndian(); // Just the declaration.
file_ops.cpp:
#include "file_ops.hpp"
// The definition.
bool systemIsLittleEndian() {
uint16_t x = 0x0011;
uint8_t *half_x = (uint8_t *) &x;
if (*half_x == 0x11)
return true;
else
return false;
}
Update
Regarding
bool MY_LIB_EXPORT someFunc();// implemented in `file_ops.cpp`
There is lots of information on the web regarding. This is a Microsoft/Windows issue. Here are couple of starting points to learn about it.
Exporting from a DLL Using __declspec(dllexport)
Importing into an Application Using __declspec(dllimport)

link functions with mismatching signature

I'm playing around with gcc and g++ compiler and trying to compile some C code within those, my purpose is to see how the compiler / linker enforces that when linking a model with some function declaration to a model with that implementation of that function, the correct function are linked ( in terms of parameters passed and values returned )
for example let's take a look at this code
#include <stdio.h>
extern int foo(int b, int c);
int main()
{
int f = foo(5, 8);
printf("%d",f);
}
after compilation within my symbol table I'd have a symbol for foo, but within the elf file format there is not place that describes the arguments taken and the function signature, ( int(int,int) ), so basically if I write some other code such as this:
char foo(int a, int b, int c)
{
return (char) ( a + b + c );
}
compile that model it'll also have some symbol called foo, what if I link these models together, what's gonna happen? I have never thought of this, and how would a compiler overcome this weakness... I know that within g++ the compiler generates some prefix for every symbol regarding to it's namespace, but does it also take in mind the signature? If anyone has ever encountered this it would be great if he could shed some light upon this problem
The problem is solved with name mangling.
In compiler construction, name mangling (also called name decoration)
is a technique used to solve various problems caused by the need to
resolve unique names for programming entities in many modern
programming languages.
It provides a way of encoding additional information in the name of a
function, structure, class or another datatype in order to pass more
semantic information from the compilers to linkers.
The need arises where the language allows different entities to be
named with the same identifier as long as they occupy a different
namespace (where a namespace is typically defined by a module, class,
or explicit namespace directive) or have different signatures (such as
function overloading).
Note the simple example:
Consider the following two definitions of f() in a C++ program:
int f (void) { return 1; }
int f (int) { return 0; }
void g (void) { int i = f(), j = f(0); }
These are distinct functions, with no relation to each other apart
from the name. If they were natively translated into C with no
changes, the result would be an error — C does not permit two
functions with the same name. The C++ compiler therefore will encode
the type information in the symbol name, the result being something
resembling:
int __f_v (void) { return 1; }
int __f_i (int) { return 0; }
void __g_v (void) { int i = __f_v(), j = __f_i(0); }
Notice that g() is mangled even though there is no conflict; name
mangling applies to all symbols.
Wow, I've kept exploring and testing it on my own and I came up with a solution which quietly amazed my mind,
so I wrote the following code and compiled it on a gcc compiler
main.c
#include <stdio.h>
extern int foo(int a, char b);
int main()
{
int g = foo(5, 6);
printf("%d", g);
return 0;
}
foo.c
typedef struct{
int a;
int b;
char c;
char d;
} mystruct;
mystruct foo(int a, int b)
{
mystruct myl;
my.a = a;
my.b = a + 1;
my.c = (char) b;
my.d = (char b + 1;
return my1;
}
now I compiled foo.c to foo.o with gcc firstly and checked the symbol table using
readelf and I had some entry called foo
also after that I compiled main.c to main.o checked the symbol table and it also had some entry called foo, I linked those two together and surprisingly it worked, I ran main.o and obviously encountered some segmentation fault, which makes sense as the actual implementation of foo as implemented in foo.o probably expects three parameters (first one should be struct adders), a parameter which isn't passed in main.o under it's definition to foo then the actual implementation accesses some memory that doesn't belong to it from the stack frame of main, then tries accessing addresses that it thought it got, and ends up with segmentation fault, that's fine,
now I compiled both models again with g++ and not gcc and what came up was amazing.. I found out that the symbol entry under foo.o was _Z3fooii and under main.o it was _Z3fooic, now my guess is that the ii suffix means int int and ic suffix means int char which probably refers to the parameters that should be passed to function hence allowing the compiler to know some function deceleration gets the actual implementation. so I changed my foo declaration in main.c to
extern int foo(int a, int b);
re-compiled and this time got the symbol _Z3fooii, I linked both models again and amazingly this time it worked, I tried running it and again encountered segmentation fault, which again also makes sense as the compiler wont always even authorize correct return values.. anyways what was my original thought - that g++ includes function signature within symbol name and thus enforces the linker to give function implementation get correct parameters to correct function declaration

Use C++11 lambdas as callbacks in GObject library

C++11 lambdas that does not capture anything can be stored in a function pointer. One just need to ensure that lambda accepts and returns the same parameters as the function pointer.
In GObject library all callbacks has type void(*GCallback) (void). This definition does not anyhow affect signature of the callback though:
The type used for callback functions in structure definitions and
function signatures. This doesn't mean that all callback functions
must take no parameters and return void. The required signature of a
callback function is determined by the context in which is used (e.g.
the signal to which it is connected). Use G_CALLBACK() to cast the
callback function to a GCallback.
In other words, one can pass function like this:
int my_function(int a, char b) {}
by casting its type (that's what G_CALLBACK do):
do_something(G_CALLBACK(my_function));
Unfortunately typecasting does not work with C++11 lambdas:
do_something(G_CALLBACK([](int a, char b) -> int {...});
// Cannot cast from type lambda to pointer type GCallback
Is it possible to use C++ lambdas of arbitrary type in place of GCallback?
UPDATE
Just to clarify, I know that lambda can be casted to a function pointer if their signatures match. My question is in another dimension.
The ISO C standard guarantees that function can be casted forth and back without loosing any precision. In other words one the following expression is valid:
int f(int a){...}
void (*void_f)() = (void (*)())f;
int (*restored_f)(int) = (int (*)(int))void_f;
restored_f(10);
My question is whether the following expression is also valid according to C++11:
int (*f)(int) = [](int a) -> int {};
void (*void_f)() = (void (*)())f;
int (*restored_f)(int) = (int (*)(int))void_f;
restored_f(10);
The following code compiles and works for me (MSVC 2013):
auto lambdaFunc = [](int a, char b) -> int { return 0; };
typedef int (*LambdaType)(int, char);
GCallback fnTest1 = G_CALLBACK((LambdaType)lambdaFunc);
GCallback fnTest2 = G_CALLBACK((LambdaType) [](int a, char b) -> int { return 0; });
do_something(fnTest1);
do_something(fnTest2);
do_something(G_CALLBACK((LambdaType)lambdaFunc));
Lambdas without a capture are implicitly convertible to a pointer to a function by the standard. Though not all compilers support this feature at the moment (https://stackoverflow.com/a/2935230/261217).
Then you can explicitly cast a function pointer to GCallback.

Changing function reference in Mac OS Process at runtime

I need to change reference of a function in a Mac OS process at runtime to a custom function defined in my own custom dylib. I kept the new function signature same as the original.
For example I need to change "open" function to "myopen" function.
I tried processing __LINKEDIT segment to get the dynamic symbol table and string table.
I used following pointers,
1. the VMAddrress from __LINKEDIT segment,
2. mach_header and vmaddr_slide from the "_dyld_register_func_for_add_image" callback,
3. symoff and stroff from symtab_command.
But I am unable to get the symbol table and string table mentioned in the __LINKEDIT segment.
Can someone throw some light on this?
Thanks in advance.
If the function in question is a library function, and not statically compiled into the executable, you don't need to do any of that - you can use function interposing, instead. Specifically, add this to your library:
// The attribute creates a Mach-O Section in your library - q.v. libgmalloc.dylib for
// a nice example
static const interpose_t interposing_functions[] \
__attribute__ ((section("__DATA, __interpose"))) = {
{ (void *)my_open, (void *)open },
{ (void *)my_close, (void *)close }, // .. etc
};
int my_open(const char *path, int flags, mode_t mode)
{
int rc;
// Prolog - do something before open
rc = open(path, flags, mode); // call real open
// Epilog - record rc, etc..
return rc;
}
There are several excellent books on OS X internals which can provide you with samples, though apparently according to S.O site policies we can't link you to them. That said, the above code snippet should work. Bear in mind, that this won't work on calls to open performed by other dylibs (though there are more complicated ways to get that, as well)

Resources