I cannot send non null type between threads in rust. I need to call a method on a NonNull pointer for the Windows rust API.
I have tried Arc<Mutex<NonNull>> and Arc<Mutex<RefCell<Box<NonNull>>> but cannot find a way to get send and sync for NonNull.
I would like the thread to halt and await the mutex, so calling a method or even mutating the NonNull type shouldnt be a threading problem but even with runtime borrow checking I get the error: 'NonNull<c_void> cannot be sent between threads safely'
and then a list of:
required because of the requirements on the impl of 'Send'
..etc.
I am about to try passing in the method as dyn but this should be possible right?
You need to provide an unsafe implementation of Send to inform the compiler that you've taken into account the thread-safety of the objects behind the pointer (which you did, since you want to use a mutex for synchronization). For example:
// Wrapper around `NonNull<RawType>` that just implements `Send`
struct WrappedPointer(NonNull<RawType>);
unsafe impl Send for WrappedPointer {}
// Safe wrapper around `WrappedPointer` that gives access to the pointer
// only with the mutex locked.
struct SafeType {
inner: Mutex<WrappedPointer>,
}
impl SafeType {
fn some_method(&self) {
let locked = self.inner.lock();
// use ptr in `locked` to implement functionality, but don't return it
}
}
Related
Consider the following code (also on the playground):
use std::{collections::HashMap, sync::Mutex};
struct MyStruct {
dummy_map: Mutex<HashMap<i64, i64>>,
}
impl MyStruct {
pub fn new() -> Self {
MyStruct {
dummy_map: Mutex::new(Default::default()),
}
}
pub fn insert(&self, key: i64, val: i64) { // <-- immutable &self
self.dummy_map.lock().unwrap().insert(key, val); // <-- insert in dummy_map
}
}
fn main() {
let s = MyStruct::new();
let key = 1;
s.insert(key, 1);
assert!(s.dummy_map.lock().unwrap().get(&key).is_some());
}
The code runs wihout panic, meaning insert takes immutable &self, but still it can insert into the map (which is wrapped in Mutex).
How come this is possible?
Would it be better for insert to take &mut self? To indicate that a field is modified...
If dummy_map is not wrapped in a Mutex, the code does not compile (as I`d expect). See this playground.
mut is kind of a misnomer in Rust, it actually means "exclusive access" (which you need to be able to mutate a value but is slightly more general). In this case, you obviously can't get exclusive access to the Mutex itself (because the whole point is to share it between threads), and therefore you can't get exclusive access to self either. However, you can get temporary exclusive access to the data inside the Mutex, at which point it becomes possible to mutate it. Kinda like what happens with Cell and RefCell too.
A Mutex is one of the Rust types that implements "interior mutability" (see the docs for Cell for discussion on that).
In short, types that implement "interior mutability" circumvent the compile-time ownership checks in favor of runtime checks. In this case, Mutex enforces the mutability rules at runtime by ensuring only one thread can access the data using its lock() and try_lock() methods.
Both locking methods return an owned MutexGuard which can provide read-only and mutable access to the data through the Deref and DerefMut traits, respectively.
In the end, this means that the variable that needs to be mut is the returned MutexGuard, not the Mutex itself.
I have an interface called Setter. Struct called SetterImpl implements this interface and has 2 setters, all of which set 2 interfaces.
type Setter interface {
A() *AInterface
B() *BInterface
}
type SetterImpl struct {
a *AInterface
b *BInterface
}
func (s *SetterImpl) A(a *AInterface) {
a = a
}
func (s *SetterImpl) B(b *AInterface) {
b = b
}
func returnSetter(a *AInterface, b *BInterface) Setter {
return &SetterImpl{a:a, b:b}
}
The setter returned by the above method is in heap(say SHeap) and sent to gRPC server. Now, I want to update a and b within the SetterImpl such that the gRPC server uses the new values.
So I have 2 goroutines now; one is gRPC server main goroutine(say MAIN), another is a forked one(say FORKED) which simply is to update the setter fields.
If I use Mutex in FORKED, that is essentially adding a fence (like java). It does not lock any variables actually(except itself).
I do not want MAIN to be able to read a and b stored in SHeap while FORKED updates them. The API threads(goroutines) in the server do not acquire Read Mutex before reading values in SHeap. So, is it even possible to do what I am trying to achieve? If yes, how do I achieve this?
I don't think you can lock variables in Go. You can only lock a code section.
m.Lock()
// do stuff
m.Unlock()
You are creating a problem for yourself out of nothing by using the "interface" vs "Impl" pattern.
You can make your life simpler by just using a regular function. Then you can put the lock inside a function and be guaranteed that it will be protected.
Since you are making an interface, you can't guarantee anything about the content of the function that implements the interface.
Variables can't be locked. So what I am trying to do is not possible.
The work-around I did was to add a RWMutex into SetterImpl and doing
mux.RLock()
defer mux.RUnlock()
in getters of a and b in SetterImpl. And when I want to set a and b in FORKED, I used
mux.Lock()
defer mux.Unlock()
This way, if a write lock is acquired by FORKED, no read lock can be acquired at that time. This is possible as the RWLock field created in SHeap is used as global variable (i.e. this mutex is never changed and passed along FORKED)
What I'm trying to do:
I have a library package which defines a few types, all implementing a given interface. Throughout the code, there are callbacks involved, and instead of writing the callback type everywhere I defined a type for it.
type Foo interface {
Bar()
}
type MyCallback func(f Foo)
type CoolFoo int
type BadFoo int
func (cf *CoolFoo) Bar(cb MyCallback) {
}
func (bf *BadFoo) Bar(cb MyCallback) {
}
Then later from client code using that library, I want to call using callbacks. If I call it by using the interface type it works:
cf := &CoolFoo{}
cf.Bar(func(f packageName.Foo) {
})
But I would rather have more self documenting code, and proper type hinting in my IDE, so I try to call it using the implementor type, such as this:
cf := &CoolFoo{}
cf.Bar(func(f packageName.CoolFoo) {
})
Which fails to compile, with the error:
cannot use func literal (type func(packageName.CoolFoo)) as type
packageName.MyCallback in argument to cf.Bar
Is it not possible, or am I making some dummy mistake ? I'm not very experienced in go dev, and I've tried looking around, but couldn't find the solution here.
What I've found is passing it as a Foo or an interface{} and then casting in the callback to what I want, but I would like to avoid it as it feels messy
Thanks for any help
The function signature is func(f Foo), so that is exactly what must be passed. The reason is simple: if your method expects a callback function of that signature, it can pass in any type that implements Foo. If you passed in a function that only accepted CoolFoo, the caller could still pass in BadFoo, because the types you've defined allow it. Thus, the compiler requires that the types match precisely. Likewise, if your callback function tries to cast to a concrete type, but that isn't the type that was passed to it, it will fail, because you're making an assumption that isn't supported by the code.
If your callback will only pass in CoolFoo, maybe that should be the type you use in the signature. If CoolFoo and BadFoo will each only pass in their respective types to their callbacks, maybe you need two different callback signatures. If they really are interchangeable, then the case you describe (a function taking CoolFoo specifically) isn't a problem with the language, it's a problem with your design.
I have code that creates a RefCell and then wants to pass a reference to that RefCell to a single thread:
use crossbeam; // 0.7.3
use std::cell::RefCell;
fn main() {
let val = RefCell::new(1);
crossbeam::scope(|scope| {
scope.spawn(|_| *val.borrow());
})
.unwrap();
}
In the complete code, I'm using a type that has a RefCell embedded in it (a typed_arena::Arena). I'm using crossbeam to ensure that the thread does not outlive the reference it takes.
This produces the error:
error[E0277]: `std::cell::RefCell<i32>` cannot be shared between threads safely
--> src/main.rs:8:15
|
8 | scope.spawn(|_| *val.borrow());
| ^^^^^ `std::cell::RefCell<i32>` cannot be shared between threads safely
|
= help: the trait `std::marker::Sync` is not implemented for `std::cell::RefCell<i32>`
= note: required because of the requirements on the impl of `std::marker::Send` for `&std::cell::RefCell<i32>`
= note: required because it appears within the type `[closure#src/main.rs:8:21: 8:38 val:&std::cell::RefCell<i32>]`
I believe I understand why this error happens: RefCell is not designed to be called concurrently from multiple threads, and since it uses internal mutability, the normal mechanism of requiring a single mutable borrow won't prevent multiple concurrent actions. This is even documented on Sync:
Types that are not Sync are those that have "interior mutability" in a non-thread-safe form, such as cell::Cell and cell::RefCell.
This is all well and good, but in this case, I know that only one thread is able to access the RefCell. How can I affirm to the compiler that I understand what I am doing and I ensure this is the case? Of course, if my reasoning that this is actually safe is incorrect, I'd be more than happy to be told why.
Another solution for this case is to move a mutable reference to the item into the thread, even though mutability isn't required. Since there can be only one mutable reference, the compiler knows that it's safe to be used in another thread.
use crossbeam; // 0.7.3
use std::cell::RefCell;
fn main() {
let mut val = RefCell::new(1);
let val2 = &mut val;
crossbeam::scope(|scope| {
scope.spawn(move |_| *val2.borrow());
})
.unwrap();
}
As bluss points out:
This is allowed because RefCell<i32> implements Send.
One way would be to use a wrapper with an unsafe impl Sync:
use crossbeam; // 0.7.3
use std::cell::RefCell;
fn main() {
struct Wrap(RefCell<i32>);
unsafe impl Sync for Wrap {};
let val = Wrap(RefCell::new(1));
crossbeam::scope(|scope| {
scope.spawn(|_| *val.0.borrow());
})
.unwrap();
}
As usual with unsafe, it is now up to you to guarantee that the inner RefCell is indeed never accessed from multiple threads simultaneously. As far as I understand, this should be enough for it not to cause a data race.
On the book The Go Programming Language Phrasebook said:
If you require performance, then you can use statically
typed definitions and avoid the dynamic lookup.
If you require flexibility, then you can use the
late binding mechanism of interfaces
Can someone explain me what "statically typed definitions" and "dynamic lookup" are for methods and functions in Go?
Imagine that we have the following code:
type A struct {}
func (a A) Foo() {
fmt.Println("Foo called")
}
type I interface {
Foo()
}
I can now create a variable of type A and call this method:
a := A{}
a.Foo()
The compiler knows the static type of the variable, so knows that the method call refers to the A.Foo method. So it can compile the above code to use a direct call to A.Foo, which will be as fast as a normal function call.
If instead we use a variable of type I, things are different:
var i I = A{}
i.Foo()
The variable i can hold any type that has a Foo method. In this particular case it is holding an A value, but won't necessarily know this at compile time. So instead the compiler generates code to check the dynamic type of i, look up the associated Foo method and finally call that method. This form of dispatch is slower than the first, but has the benefit the code will work for any type implementing the interface.
This is similar to C++'s distinction between virtual and non-virtual methods, except rather than the type of dispatch being fixed for a method at its definition, it depends on the type of variable you use in Go.
What the book refers to when it says using static types is using non interface types:
func Foo(v int, s string) { ... }
The other option is using interface:
func Bar(a interface{}, b interface{}) { ... }
Because with the first option, Go will know at compile time what type of value the function will retrieve (int and string in this case), it will compile the code specifically for those types.
With the second option, you will have to use reflection at runtime in order to know the values contained in the interfaces. This is a bit of overhead, but it allows you to pass different types of values as parameters to the function, thus being more dynamic.
Further reading: Laws of Reflection in Go