Link Error during Rust compilation (Cargo) - winapi

I'm calling some WinAPI functions with the Rust FFI (in this case MessageBoxA).
My code was working until I did a little variable change and the compilation gave me an error as:
= note: Non-UTF-8 output: WinAPI-dd8845a539e186b8.4ojwfrbxjnkzuhga.rcgu.o : er
ror LNK2019: symbole externe non r\xe9solu MessageBoxA r\xe9f\xe9renc\xe9 dans l
a fonction _ZN6WinAPI4main17hdf93991da0bc3966E\r\nd:\\core\\Confidential\\Forens
ic\\Rust\\WinAPI\\target\\debug\\deps\\WinAPI-dd8845a539e186b8.exe : fatal error
LNK1120: 1 externes non r\xe9solus\r\n
The last line is in french and it means LNK1120: 1 unresolved external.
I kind of understand that is an encoding error but I have no ideas how to solve it.
So I canceled the little changes I did in my code but it keep displaying that weird message (The error message is actually bigger but not comprehensible).
It's actually a cargo project, the code if you want to check it:
#[cfg(windows)]
#[link(name = "user32", kind = "dylib")]
extern crate libc;
mod ffi{
use libc::{c_uint,uintptr_t,c_void};
type HANDLE = *mut c_void;
pub type UINT = c_uint;
pub type UINT_PTR = uintptr_t;
pub type HWND = HANDLE;
pub type LPCTSTR = *const i8;
pub const MB_OK: u32 = 0x0;
pub const MB_OKCANCEL: u32 = 0x00000001;
pub const MB_ICONWARNING: u32 = 0x00000030;
pub const MB_ICONINFORMATION: u32 = 0x00000040;
pub const MB_ICONQUESTION: u32 = 0x00000020;
}
extern "system"{
fn MessageBoxA(hWnd: ffi::HWND, lpText: ffi::LPCTSTR, lpCaption: ffi::LPCTSTR, uType: u32) -> u32;
}
use ffi::LPCTSTR;
use ffi::MB_OK;
use ffi::MB_ICONINFORMATION;
fn main() -> std::io::Result<()>{
unsafe{
let buffer: &[u8] = &[97,99,107,101,0]; // "acke" as a null terminated str
let lpData: LPCTSTR = core::str::from_utf8_unchecked(buffer).as_ptr() as *const i8;
let lpCaption: LPCTSTR = "Information".as_ptr() as *const i8;
MessageBoxA(
std::ptr::null_mut(),
lpData,
lpCaption,
MB_OK | MB_ICONINFORMATION,
);
};
return Ok(());
}
#[cfg(not(windows))]
fn main() -> std::io::Result<()>{
println!("That program only runs on Windows 10 architectures.");
return Ok(());
}
Important : The error doesn't occure when I put the call to MessageBoxA in comment.

I changed my Visual Studio default language to English because of encoding problems with link.exe.
Then I got an error about unresolved external: MessageBoxA#16. To solve that, I moved the line extern "system"{ /* Functions prototypes */ } directly after #[link(name="user32")] declaration.
You also need Windows 10 SDK installed.
Thanks to rustup.exe for giving really good installation indications!

Related

Rust function pointer to winapi LPVOID

I am doing a CTF and for that I would like to use the ms detours library in rust. I found rust bindings to the detours lib (detours-sys). DetourAttach call wants to receive a pointer to a pointer to a function as first parameter.
fn hook_cwtick() {
unsafe {
let module_name = "GameLogic.dll";
// this returns the correct value I checked in x32dbg
let func_loc: CwTickFn = hook_game_func(module_name, "GameAPI::Tick(float)");
{
let str_format = format!("func address: {:p}", func_loc);
MessageBoxW(null_mut(), win32_string(&str_format).as_ptr(), win32_string("Function location").as_ptr(), MB_OK);
}
let c_like_func = (std::mem::transmute::<CwTickFn, c_void>(func_loc));
let c_like_hooked_func = std::mem::transmute::<CwTickFn, c_void>(hookedCwTick);
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
// this returns the compiler error
DetourAttach(c_like_func, c_like_hooked_func);
DetourTransactionCommit();
}
}
The error I get is the following:
error[E0308]: mismatched types
--> src\lib.rs:75:22
|
75 | DetourAttach(c_like_func, c_like_hooked_func);
| ^^^^^^^^^^^ expected *-ptr, found enum `winapi::c_void`
|
= note: expected raw pointer `*mut *mut winapi::c_void`
found enum `winapi::c_void
This is probably some rookie rust mistake. I have tried to double dereference the c_like_func and passing it like **c_like_func but the compiler complained that it cannot be dereferenced.
Any input appreciated

Why does calling SystemParametersInfo from Rust to set the wallpaper set it to black?

I am trying to set the Windows background in Rust using the winapi crate and SystemParametersInfo, but it sets the background to black. In C++, that usually means that pvParam isn't passed correctly or it has the wrong type.
What's wrong?
#[cfg(windows)]
extern crate winapi;
use winapi::ctypes::c_void;
use winapi::um::winuser::{SystemParametersInfoA, SPIF_UPDATEINIFILE, SPI_SETDESKWALLPAPER};
fn main() {
let mut image_path = "Path to Image";
let image_path_c_ptr: *mut c_void = &mut image_path as *mut _ as *mut c_void;
unsafe {
SystemParametersInfoA(
SPI_SETDESKWALLPAPER,
0,
image_path_c_ptr,
SPIF_UPDATEINIFILE,
);
}
}
Rust strings are not C strings. You should instead use CString to interface with C code:
use std::ffi::CString;
// use ...
fn main() {
let mut image_path = CString::new("Path to Image").unwrap();
unsafe {
SystemParametersInfoA(
SPI_SETDESKWALLPAPER,
0,
image_path.as_ptr() as *mut c_void,
SPIF_UPDATEINIFILE,
);
}
}
To elaborate: image_path is a &str (a fat pointer). By taking a mutable reference to it you are getting a &mut &str. You then pass it to C, which will dereference the pointer and get a &str.
But C code does not know how to deal with a Rust type: it is only aware of C strings and instead expects a pointer to the first byte. It also expects the string to be NUL terminated, which Rust strings are not. Thus it makes no sense to pass a Rust &str to C code in this case and this is exactly the reason CStr and CString exist.
This was my final working code:
#[cfg(windows)]
extern crate winapi;
use std::ffi::CString;
use winapi::ctypes::c_void;
use winapi::um::winuser::{SystemParametersInfoA, SPIF_UPDATEINIFILE, SPI_SETDESKWALLPAPER};
fn main() {
let mut image_path = CString::new("Path to Image").unwrap();
unsafe {
SystemParametersInfoA(
SPI_SETDESKWALLPAPER,
0,
image_path.as_ptr() as *mut c_void,
SPIF_UPDATEINIFILE,
);
}
}

Unresolved external symbol AES_set_encrypt_key when linking Rust code to boringssl on Windows

I'm trying to compile my library wrapper of boringssl. I've compiled the crypto.lib file that definitely contains the symbol AES_set_encrypt_key, judging by the command:
nm crypto.lib --demangle
Then I've generated a FFI file that declares that symbol:
use std::os::raw::{c_char, c_int, c_uchar, c_uint, c_void};
#[repr(C)]
pub struct aes_key_st {
pub rd_key: [u32; 60usize],
pub rounds: c_uint,
}
pub type AES_KEY = aes_key_st;
extern "C" {
pub fn AES_set_encrypt_key(key: *const u8, bits: c_uint, aeskey: *mut AES_KEY) -> c_int;
pub fn foo() -> c_int;
}
Here are the original C declarations:
struct aes_key_st {
uint32_t rd_key[4 * (AES_MAXNR + 1)];
unsigned rounds;
};
typedef struct aes_key_st AES_KEY;
OPENSSL_EXPORT int AES_set_encrypt_key(const uint8_t *key, unsigned bits, AES_KEY *aeskey);
build.rs:
fn main() {
let out_dir = "<path_to_lib>"
println!("cargo:rustc-link-search=native={}", out_dir);
println!("cargo:rustc-link-lib=static=crypto");
}
I am getting an LNK2019 (unresolved external symbol) error when compiling this code.
Why is this happening?
I had compiled my library with 32-bit version of MSVC, but Rust compiles programs with 64-bit one. That's why there was a conflict.

How to use SetWindowsHookEx in Rust?

I am trying to figure out how to set global Windows hooks inside of Rust. I can find multiple examples for other languages, but there doesn't seem to be anything for Rust.
What I managed to get so far:
extern crate user32;
extern crate winapi;
const WH_KEYBOARD_LL: i32 = 13;
fn main() {
let hook_id = user32::SetWindowsHookExA(
WH_KEYBOARD_LL,
Some(hook_callback),
// No idea what goes here ,
0,
);
}
fn hook_callback(code: i32, wParam: u64, lParam: i64) -> i64 {
// ...
}
The compiler complains that it needs a "system" fn for the callback function, but is getting a Rust fn, which makes sense, but I still don't know how to make that work.
From what I gathered from the documentation, the third parameter hMod should point to the same module that has the callback function, and the examples in other languages uses some function that gets the current module handle, but I don't know how to do that in Rust.
The compiler complains that it needs a "system" fn for the callback function, but is getting a Rust fn, which makes sense, but I still don't know how to make that work.
The compiler actually gives you exactly what you need ... if you continue reading the error you'll see:
expected type `unsafe extern "system" fn(i32, u64, i64) -> i64`
found type `fn(i32, u64, i64) -> i64 {hook_callback}`
Adding that, gives:
extern "system" fn hook_callback(code: i32, wParam: u64, lParam: i64) -> i64 {
0
}
From what I gathered from the documentation, the third parameter hMod should point to the same module that has the callback function, and the examples in other languages uses some function that gets the current module handle, but I don't know how to do that in Rust.
Again, reading further into the WinAPI documentation shows that NULL should be the value of this parameter if the thread ID (the last argument) specifies that its within the same process. Since you've passed zero - which the documentation states is associated with all threads in the current process ... that's what it should be... NULL. So now we get:
let hook_id =
user32::SetWindowsHookExA(WH_KEYBOARD_LL, Some(hook_callback), std::ptr::null_mut(), 0);
This compiles.
Accounting for the other errors around unsafe that you'll get ... this gives you (full working code):
extern crate user32;
extern crate winapi;
const WH_KEYBOARD_LL: i32 = 13;
fn main() {
unsafe {
let hook_id =
user32::SetWindowsHookExA(WH_KEYBOARD_LL, Some(hook_callback), std::ptr::null_mut(), 0);
// Don't forget to release the hook eventually
user32::UnhookWindowsHookEx(hook_id);
}
}
extern "system" fn hook_callback(code: i32, wParam: u64, lParam: i64) -> i64 {
0
}

What's wrong with this Rust FFI declaration of CertOpenSystemsStoreW or CertCloseStore?

I'm trying to call the Win32 API CertOpenSystemsStoreW and CertCloseStore functions from Rust. When I do, I get an access violation on CertCloseStore, so I suppose I have the size wrong on some argument type, but I can't see it.
The following Python code works (and I have equivalent working C++, but not as nicely contained):
In [1]: import ctypes
In [2]: c32 = ctypes.windll.crypt32
In [3]: c32.CertOpenSystemStoreW.argtypes = [ctypes.c_void_p, ctypes.c_wchar_p]
In [4]: c32.CertOpenSystemStoreW.restype = ctypes.c_void_p
In [5]: c32.CertCloseStore.argtypes=[ctypes.c_void_p, ctypes.c_ulong]
In [6]: s = c32.CertOpenSystemStoreW(0, "my")
In [7]: c32.CertCloseStore(s, 0)
Out[7]: 1
Here's the failing Rust:
extern crate libc;
use libc::{c_ulong, c_int, c_void};
use std::ffi::OsStr;
use std::os::windows::ffi::OsStrExt;
use std::ptr::null;
type HPROVIDER = c_void;
type HCERTSTORE = c_void;
type BOOL = c_int;
#[link(name = "Crypt32")]
extern "stdcall" {
fn CertOpenSystemStoreW(
hProv: *const HPROVIDER, szSubsystemProtocol: *const u16) -> HCERTSTORE;
fn CertCloseStore(
hCertStore: HCERTSTORE, dwFlags: c_ulong) -> BOOL;
}
fn to_utf16(s: &str) -> Vec<u16> {
let os_s = OsStr::new(s);
return os_s.encode_wide().chain(Some(0).into_iter()).collect::<Vec<_>>();
}
fn main() {
let protocol_utf16 = to_utf16("my");
let storehandle;
unsafe {
storehandle = CertOpenSystemStoreW(null(), protocol_utf16.as_ptr());
}
let freeresults;
unsafe {
freeresults = CertCloseStore(storehandle, 0);
}
println!("{}", freeresults);
}
I'm using Rust 1.16.
Well, there were two problems:
DWORDs are 32 bits, no matter whether you're on 64bit or 32bit windows (which makes sense, I suppose). So my CertCloseStore's second argument is wrong.
c_void is not a pointer type - it's just a u8. So my code above should be something like type HPROVIDER = *const c_void; (Which is not great, since it makes all HPROVIDERs const, but I don't see a way to do a Rust-style pointer typedef without also specifying either 'mut' or 'const').

Resources