I’m working with a native linux C binary which has a fairly expensive initialization call which I would like to perform once at application startup. This call should open a bunch of file handles internally for later use. When I call this expensive initialization C function from Go, it completes successfully and correctly opens the files but those handles are open only for the duration of the call to the C function! This means that when I call successive C functions against the same library from Go, the file handles are no longer open and the calls fail. I have verified this using the lsof command. Interestingly, when the initialization call as well as calls to subsequent behavior are composed into a single C function which is then called from Go, the files are opened and remain open, allowing successful completion of all desired functionality.
Is there some kind of undocumented cgo behavior which is “cleaning up”, shutting down, or even leaking file handles or other stateful resources between multiple invocations of C functions from Go? If so, is this behavior configurable? We don’t have access to the source code for this library.
Also, I've verified that this is not related to thread-local storage. Calling runtime.LockOSThread() has no effect and we've verified that the files are closed after control returns from C back to the calling Go code.
Here’s an example of the kind of Go code I’d like to write:
// Go code:
func main() {
C.Initialize()
C.do_stuff() // internal state is already cleaned up! This call fails as a result. :(
}
Here’s an example of a C function that invokes the initialization and behavior all at once. This “wrapping” function is invoked from Go:
// C code:
void DoEverything(void)
{
Initialize();
do_stuff(); // succeeds because all internal state is intact (not cleaned up).
}
Ok, this is a bit embarrassing, but I figured it out. Right after calling initialize(), I was calling defer close(), but it was actually defer fmt.Println(close()). Because arguments to deferred functions are resolved immediately (not deferred), the close function was being invoked before we could invoke any other behavior. The golang blog clearly explains argument resolution to deferred function calls.
Related
I’m trying to write some Rust code that uses Windows.Web.UI.Interop.WebViewControl (which is a Universal Windows Platform out-of-process wrapper expressly designed so Win32 apps can use EdgeHTML), and it’s all compiling, but not working properly at runtime.
The relevant code boils down to this, using the winit, winapi and winrt crates:
use winit::os::windows::WindowExt;
use winit::{EventsLoop, WindowBuilder};
use winapi::winrt::roapi::{RoInitialize, RO_INIT_SINGLETHREADED};
use winapi::shared::winerror::S_OK;
use winrt::{RtDefaultConstructible, RtAsyncOperation};
use winrt::windows::foundation::Rect;
use winrt::windows::web::ui::interop::WebViewControlProcess;
fn main() {
assert!(unsafe { RoInitialize(RO_INIT_SINGLETHREADED) } == S_OK);
let mut events_loop = EventsLoop::new();
let window = WindowBuilder::new()
.build(&events_loop)
.unwrap();
WebViewControlProcess::new()
.create_web_view_control_async(
window.get_hwnd() as usize as i64,
Rect {
X: 0.0,
Y: 0.0,
Width: 800.0,
Height: 600.0,
},
)
.expect("Creation call failed")
.blocking_get()
.expect("Creation async task failed")
.expect("Creation produced None");
}
The WebViewControlProcess instantiation works, and the CreateWebViewControlAsync function does seem to care about the value it received as host_window_handle (pass it 0, or one off from the actual HWND value, and it complains). Yet the IAsyncOperation stays determinedly at AsyncStatus.Started (0), and so the blocking_get() call hangs indefinitely.
A full, runnable demonstration of the issue (with a bit more instrumentation).
I get the feeling that the WebViewControlProcess is at fault: its ProcessId is stuck at 0, and it doesn’t look to have spawned any subprocess. The ProcessExited event does not seem to be being fired (I attached something to it immediately after instantiation, is there opportunity for it to be fired before that?). Calling Terminate() fails as one might expect in such a situation, E_FAIL.
Have I missed some sort of initialization for using Windows.Web.UI.Interop? Or is there some other reason why it’s not working?
It turned out that the problem was threading-related: the winit crate was doing its event loop in a different thread, and I did not realise this; I had erroneously assumed winit to be a harmless abstraction, which it turned out not quite to be.
I discovered this when I tried minimising and porting a known-functioning C++ example, this time doing all the Win32 API calls manually rather than using winit, so that the translation was correct. I got it to work, and discovered this:
The IAsyncOperation is fulfilled in the event loop, deep inside a DispatchMessageW call. That is when the Completion handler is called. Thus, for the operation to complete, you must run an event loop on the same thread. (An event loop on another thread doesn’t do anything.) Otherwise, it stays in the Started state.
Fortunately, winit is already moving to a new event loop which operates in the same thread, with the Windows implementation having landed a few days ago; when I migrated my code to use the eventloop-2.0 branch of winit, and to using the Completed handler instead of blocking_get(), it all started working.
I shall clarify about the winrt crate’s blocking_get() call which would normally be the obvious solution while prototyping: you can’t use it in this case because it causes deadlock, since it blocks until the IAsyncOperation completes, but the IAsyncOperation will not complete until you process messages in the event loop (DispatchMessageW), which will never happen because you’re blocking the thread.
Try to initialize WebViewProcessControl with winrt::init_apartment(); And it may needs a single-threaded apartment(according to the this answer).
More attention on Microsoft Edge Developer Guide:
Lastly, power users might notice the apppearance of the Desktop App
Web Viewer (previously named Win32WebViewHost), an internal system app
representing the Win32 WebView, in the following places:
● In the Windows 10 Action Center. The source of these notifications
should be understood as from a WebView hosted from a Win32 app.
● In the device access settings UI
(Settings->Privacy->Camera/Location/Microphone). Disabling any of
these settings denies access from all WebViews hosted in Win32 apps.
var p = &sync.Pool{
New: func() interface{} {
return &serveconn{}
},
}
func newServeConn() *serveconn {
sc := p.Get().(*serveconn)
runtime.SetFinalizer(sc, (*serveconn).finalize)
fmt.Println(sc, "SetFinalizer")
return sc
}
func (sc *serveconn) finalize() {
fmt.Println(sc, "finalize")
*sc = serveconn{}
runtime.SetFinalizer(sc, nil)
p.Put(sc)
}
The above code tries to reuse object by SetFinalizer, but after debug I found finalizer is never called, why?
UPDATE
This may be related:https://github.com/golang/go/issues/2368
The above code tries to reuse object by SetFinalizer, but after debug I found finalizer is never called, why?
The finalizer is only called on an object when the GC
marks it as unused and then tries to sweep (free) at the end
of the GC cycle.
As a corollary, if a GC cycle is never performed during the runtime of your program, the finalizers you set may never be called.
Just in case you might hold a wrong assumption about the Go's GC, it may worth noting that Go does not employ reference counting on values; instead, it uses GC which works in parallel with the program, and the sessions during which it works happen periodically and are triggered by certain parameters like pressure on the heap produced by allocations.
A couple assorted notes regarding finalizers:
When the program terminates, no GC is forcibly run.
A corollary of this is that a finalizer is not guaranteed
to run at all.
If the GC finds a finalizer on an object about to be freed,
it calls the finalizer but does not free the object.
The object itself will be freed only at the next GC cycle —
wasting the memory.
All in all, you appear as trying to implement destructors.
Please don't: make your objects implement the sort-of standard method called Close and state in the contract of your type that the programmer is required to call it when they're done with the object.
When a programmer wants to call such a method no matter what, they use defer.
Note that this approach works perfectly for all types in the Go
stdlib which wrap resources provided by the OS—file and socket descriptors. So there is no need to pretend your types are somehow different.
Another useful thing to keep in mind is that Go was explicitly engineered to be no-nonsense, no-frills, no-magic, in-your-face language, and you're just trying to add magic to it.
Please don't, those who like decyphering layers of magic do program in Scala different languages.
We hit a case where it would be the best solution for us to put a FreeLibrary call into DllMain / DLL_PROCESS_DETACH.
Of course, you must not do that:
It is not safe to call FreeLibrary from DllMain.
The use case is that we have a situation like this:
(unknown client dll or exe) links dynamically or statically to ->
-> DLL_1, loads dynamically -> DLL_x
DLL_1 should load DLL_x transparently wrt. to it's client code, and it should to load DLL_x dynamically. Now, the loading can be done lazily, so that the LoadLibrary call needn't reside in the DLL_PROCESS_ATTACH part of DLL_1.
But once the client is done with DLL_1, when/before DLL_1 is unloaded from the process, it should also unload (== FreeLibrary) DLL_x.
Is there any way to do this without an explicit DLL_1/Uninitialize function that must be called by the client?
I'll note:
DllMain, and thus also any C++ global static destructor cannot be used.
Is there any other callback mechanism in either kernel32/ntdll or maybe in the shared MS CRT to make this happen?
Are there other patterns to make this usecase work?
The correct approach is an explicit Uninitialize function in DLL_1.
However, if you can't do that, you can work around the problem by launching a helper thread to do the unload for you. If you want to play it safe, launch the thread at the same time you load DLL_x and have it wait on an event object. (For the record, though, it is generally considered safe to launch a thread from DllMain so long as you respect the fact that it won't start up until DllMain has exited.)
Obviously, the helper thread's code can't be in DLL_1. If you can modify DLL_x you can put it there. If not, you'll need a helper DLL. In either case, the DLL containing the helper thread's code can safely self-unload using the FreeLibraryAndExitThread function.
I guess this is a really nasty issue - seems like one of the property destructors of my class creates deadlock. Property destructors are called automatically after class destructor. And I'd like to call them manually and make a log entry after every single one succeeds.
The problem only occurs on devices, where debugger can't be used, so I am using log instead.
Client::~Client() {
// Stops io service and disconnects sockets
exit();
LOG("io_service stopped"<<endl);
// Destroy IO service
io_.~io_service();
LOG("io_service destroyed"<<endl);
}
But the code above actually causes exception, because the ~io_service() gets called twice.
So is there a way to do this properly? If not, what's an alternative to debugging destructors?
You can't alter the compiler behaviour like that. the compiler will augment the destructor to destruct nested objects.
What you can do is to declare io as a pointer and allocate it dynamically with new. then call delete io and monitor what happens there.
Other solution is just to put a breakpoint on the io destructor and follow what happens there upon destruction. this is probably the best idea.
I'm developing a launcher for a game.
Want to intercept game's call for a function that prints text.
I don't know whether the code that contains this function is dynamically linked or statically. So I dont even know the function name.
I did intercepted some windows-api calls of this game through microsoft Detours, Ninject and some others.
But this one is not in import table either.
What should I do to catch this function call? What profiler should be used? IDA? How this could be done?
EDIT:
Finally found function address. Thanks, Skino!
Tried to hook it with Detours, injected dll. Injected DllMain:
typedef int (WINAPI *PrintTextType)(char *, int, float , int);
static PrintTextType PrintText_Origin = NULL;
int WINAPI PrintText_Hooked(char * a, int b, float c, int d)
{
return PrintText_Origin(a, b, c , d);
}
HMODULE game_dll_base;
/* game_dll_base initialization goes here */
BOOL APIENTRY DllMain(HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved)
{
if(fdwReason==DLL_PROCESS_ATTACH)
{
DisableThreadLibraryCalls(hinstDLL);
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
PrintText_Origin = (PrintTextType)((DWORD)game_dll_base + 0x6049B0);
DetourAttach((PVOID *)&PrintText_Origin , PrintText_Hooked);
DetourTransactionCommit();
}
}
It hooks as expected. Parameter a has text that should be displayed. But when calling original function return PrintText_Origin (a, b, c , d); application crashes(http://i46.tinypic.com/ohabm.png, http://i46.tinypic.com/dfeh4.png)
Original function disassembly:
http://pastebin.com/1Ydg7NED
After Detours:
http://pastebin.com/eM3L8EJh
EDIT2:
After Detours:
http://pastebin.com/GuJXtyad
PrintText_Hooked disassembly http://pastebin.com/FPRMK5qt w3_loader.dll is the injected dll
Im bad at ASM, please tell what can be wrong ?
Want to intercept game's call for a function that prints text.
You can use a debugger for the investigative phase. Either IDA, or even Visual Studio (in combination with e.g. HxD), should do. It should be relatively easy to identify the function using the steps below:
Identify a particular fragment of text whose printing you want to trace (e.g. Hello World!)
Break the game execution at any point before the game normally prints the fragment you identified above
Search for that fragment of text† (look for either Unicode or ANSI) in the game's memory. IDA will allow you to do that IIRC, as will the free HxD (Extras > Open RAM...)
Once the address of the fragment has been identified, set a break-on-access/read data breakpoint so the debugger will give you control the moment the game attempts to read said fragment (while or immediately prior to displaying it)
Resume execution, wait for the data breakpoint to trigger
Inspect the stack trace and look for a suitable candidate for hooking
Step through from the moment the fragment is read from memory until it is printed if you want to explore additional potential hook points
†provided text is not kept compressed (or, for whatever reason, encrypted) until the very last moment
Once you are done with the investigative phase and you have identified where you'd like to inject your hook, you have two options when writing your launcher:
If, based on the above exercise, you were able to identify an export/import after all, then use any API hooking techniques
EDIT Use Microsoft Detours, making sure that you first correctly identify the calling convention (cdecl, fastcall, stdcall) of the function you are trying to detour, and use that calling convention for both the prototype of the original as well as for the implementation of the dummy. See examples.
If not, you will have to
use the Debugging API to programatically load the game
compute the hook address based on your investigative phase (either as a hard-coded offset from the module base, or by looking for the instruction bytes around the hook site‡)
set a breakpoint
resume the process
wait for the breakpoint to trigger, do whatever you have to do
resume execution, wait for the next trigger etc. again, all done programatically by your launcher via the Debugging API.
‡to be able to continue to work with eventual patch releases of the game
At this stage it sounds like you don't have a notion of what library function you're trying to hook, and you've stated it's not (obviously at least) an imported external function in the import table which probably means that the function responsible for generating the text is likely located inside the .text of the application you are disassembling directly or loaded dynamically, the text generation (especially in a game) is likely a part of the application.
In my experience, this simplest way to find code that is difficult to trace such as this is by stopping the application shortly during or before/after text is displayed and using IDA's fabulous call-graph functionality to establish what is responsible for writing it out (use watches and breakpoints liberally!)
Look carefully to calls to CreateRemoteThread or any other commonly used dynamic loading mechanism if you have reason to believe this functionality might be provided by an exported function that isn't showing up in the import table.
I strongly advice against it but for the sake of completeness, you could also hook NtSetInformationThread in the system service dispatch table. here's a good dump of the table for different Windows versions here. If you want to get the index in the table yourself you can just disassemble the NtSetInformationThread export from ntdll.dll.