I am trying to write a chip8 emulator and the borrow checker is giving me a hard time.
The idea is to decode an opcode through looking up a method pointer inside a HashMap and then executing this method pointer but I cannot get the mutable method pointers to work correctly:
struct Chip8 {
opcode: u16,
//... other fields
jump_table: HashMap<u16, Box<fn(&mut Chip8)>>,
}
Function using the pointers:
fn execute_decoded(&mut self, key: u16) {
let func = self.jump_table.get(&key);
match func {
Some(func) => func(self),
None => {
println!("invalid op: {}", self.opcode);
sleep(Duration::from_millis(10000));
return;
}
}();
self.program_counter = self.program_counter + 2;
}
The checker complains:
cannot borrow `*self` as mutable because `self.jump_table` is also borrowed as immutable
--> main.rs:168:36
|
165 | let func = self.jump_table.get(&key);
| --------------- immutable borrow occurs here
...
168 | Some(func) => func(self),
| ^^^^ mutable borrow occurs here
...
178 | }
| - immutable borrow ends here
I do not understand why this error is happening.
Why is self.jump_table.get(&key) borrowing at all? Based on the signature of execute_decoded, I was assuming that it works on a mutable borrowed version of self and no additional borrowing is needed.
There's no reason to Box the function pointers in the HashMap, that only introduces unneeded indirection.
As has already been mentioned, you are borrowing the function pointer. The thing is, there's no reason to. You can just copy the function pointer to disassociate it from the HashMap:
use std::collections::HashMap;
struct Chip8 {
jump_table: HashMap<u16, fn(&mut Chip8)>,
}
impl Chip8 {
fn execute_decoded(&mut self, key: u16) {
let func = self.jump_table.get(&key).map(|x| *x);
match func {
Some(func) => func(self),
None => {
println!("invalid op");
}
};
}
}
fn main() {}
A HashMap in Rust owns everything inside of it. In order to get your function pointer you are borrowing it with let func = self.jump_table.get(&key);. So now, func is immutably borrowing self.jump_table (which is an element of self).
The issue is that you are then trying to pass all of self into func. This would be fine if you were passing in self immutably, as you can borrow self immutably as many times as you want. However, since you are trying to mutably borrow self the compiler will not allow you to do so since you have just immutably borrowed a portion of self (specifically self.jump_table).
One way to fix this is to split up your Chip8 struct into smaller structs, such that you can pass all of the necessary information into func without also passing in jump_table.
Related
There is already a very popular question about this topic but I don;t fully understand the answer.
The goal is:
I need a list (read a Vec) of "function pointers" that modify data stored elsewhere in a program. The simplest example I can come up with are callbacks to be called when a key is pressed. So when any key is pressed, all functions passed to the object will be called in some order.
Reading the answer, it is not clear to me how I would be able to make such a list. It sounds like I would need to restrict the type of the callback to something known, else I don't know how you would be able to make an array of it.
It's also not clear to me how to store the data pointers/references.
Say I have
struct Processor<CB>
where
CB: FnMut(),
{
callback: CB,
}
Like the answer suggests, I can't make an array of processors, can I? since each Processor is technically a different type depending on the generic isntantiation.
Indeed, you can't make a vector of processors. Usually, closures all have different, innominable types. What you want instead are trait objects, which allow you to have dynamic dispatch of callback calls. Since those are not Sized, you'd probably want to put them in a Box. The final type is Vec<Box<dyn FnMut()>>.
fn add_callback(list: &mut Vec<Box<dyn FnMut()>>, cb: impl FnMut() + 'static) {
list.push(Box::new(cb))
}
fn run_callback(list: &mut [Box<dyn FnMut()>]) {
for cb in list {
cb()
}
}
see the playground
If you do like that, however, you might have some issues with the lifetimes (because your either force to move-in everything, or only modify values that life for 'static, which isn't very convenient. Instead, the following might be better
#[derive(Default)]
struct Producer<'a> {
list: Vec<Box<dyn FnMut() + 'a>>,
}
impl<'a> Producer<'a> {
fn add_callback(&mut self, cb: impl FnMut() + 'a) {
self.list.push(Box::new(cb))
}
fn run_callbacks(&mut self) {
for cb in &mut self.list {
cb()
}
}
}
fn callback_1() {
println!("Hello!");
}
fn main() {
let mut modified = 0;
let mut prod = Producer::default();
prod.add_callback(callback_1);
prod.add_callback(
|| {
modified += 1;
println!("World!");
}
);
prod.run_callbacks();
drop(prod);
println!("{}", modified);
}
see the playground
Just a few things to note:
You manually have to drop the producer, otherwise Rust will complain that it will be dropped at the end of the scope, but it contains (through the closure) an exclusive reference to modified, which is not ok since I try to read it.
Current, run_callbacks take a &mut self, because we only require for a FnMut. If you wanted it to be only a &self, you'd need to replace FnMut with Fn, which means the callbacks can still modify things outside of them, but not inside.
Yes, all closures are differents type, so if you want to have a vec of different closure you will need to make them trait objects. This can be archieve with Box<dyn Trait> (or any smart pointer). Box<dyn FnMut()> implements FnMut(), so you can have Processor<Box<dyn FnMut()>> and can make a vec of them, and call the callbacks on them: playground
I want to solve a leetcode question in Rust (Remove Nth Node From End of List). My solution uses two pointers to find the Node to remove:
#[derive(PartialEq, Eq, Debug)]
pub struct ListNode {
pub val: i32,
pub next: Option<Box<ListNode>>,
}
impl ListNode {
#[inline]
fn new(val: i32) -> Self {
ListNode { next: None, val }
}
}
// two-pointer sliding window
impl Solution {
pub fn remove_nth_from_end(head: Option<Box<ListNode>>, n: i32) -> Option<Box<ListNode>> {
let mut dummy_head = Some(Box::new(ListNode { val: 0, next: head }));
let mut start = dummy_head.as_ref();
let mut end = dummy_head.as_ref();
for _ in 0..n {
end = end.unwrap().next.as_ref();
}
while end.as_ref().unwrap().next.is_some() {
end = end.unwrap().next.as_ref();
start = start.unwrap().next.as_ref();
}
// TODO: fix the borrow problem
// ERROR!
// start.unwrap().next = start.unwrap().next.unwrap().next.take();
dummy_head.unwrap().next
}
}
I borrow two immutable references of the linked-list. After I find the target node to remove, I want to drop one and make the other mutable. Each of the following code examples leads to a compiler error:
// ERROR
drop(end);
let next = start.as_mut().unwrap.next.take();
// ERROR
let mut node = *start.unwrap()
I don't know if this solution is possible to be written in Rust. If I can make an immutable reference mutable, how do I do it? If not, is there anyway to implement the same logic while making the borrow checker happy?
The correct answer is that you should not be doing this. This is undefined behavior, and breaks many assumptions made by the compiler when compiling your program.
However, it is possible to do this. Other people have also mentioned why this is not a good idea, but they haven't actually shown what the code to do something like this would look like. Even though you should not do this, this is what it would look like:
unsafe fn very_bad_function<T>(reference: &T) -> &mut T {
let const_ptr = reference as *const T;
let mut_ptr = const_ptr as *mut T;
&mut *mut_ptr
}
Essentially, you convert a constant pointer into a mutable one, and then make the mutable pointer into a reference.
Here's one example why this is very unsafe and unpredictable:
fn main() {
static THIS_IS_IMMUTABLE: i32 = 0;
unsafe {
let mut bad_reference = very_bad_function(&THIS_IS_IMMUTABLE);
*bad_reference = 5;
}
}
If you run this... you get a segfault. What happened? Essentially, you invalidated memory rules by trying to write to an area of memory that had been marked as immutable. Essentially, when you use a function like this, you break the trust the compiler has made with you to not mess with constant memory.
Which is why you should never use this, especially in a public API, because if someone passes an innocent immutable reference to your function, and your function mutates it, and the reference is to an area of memory not meant to be written to, you'll get a segfault.
In short: don't try to cheat the borrow checker. It's there for a reason.
EDIT: In addition to the reasons I just mentioned on why this is undefined behavior, another reason is breaking reference aliasing rules. That is, since you can have both a mutable and immutable reference to a variable at the same time with this, it causes loads of problems when you pass them in separately to the same function, which assumes the immutable and mutable references are unique. Read this page from the Rust docs for more information about this.
Is there a way to make an immutable reference mutable?
No.
You could write unsafe Rust code to force the types to line up, but the code would actually be unsafe and lead to undefined behavior. You do not want this.
For your specific problem, see:
How to remove the Nth node from the end of a linked list?
How to use two pointers to iterate a linked list in Rust?
The below code won't compile because the compiler thinks I shouldn't assign to t1 since it is borrowed, but in reality the function always_returns_no_lifetime will always be returning a variant of the enum that actually doesn't have an lifetime, so it is okay for me to modify t1. How can I get the compiler to understand this or how should I reorganize my code to make this error not happen?
#[derive(Clone)]
enum Types<'a> {
NoLifetime(i32),
AlsoNoLifetime(i32),
AlsoAlsoNoLifetime(i32),
HasLifetime(&'a str)
}
fn always_returns_no_lifetime<'a>(some_type: &'a Types) -> Types<'a> {
match *some_type {
Types::HasLifetime(text) => panic!("I only return the type that has no lifetime"),
_ => some_type.clone()
}
}
fn main() {
let mut t1 = Types::NoLifetime(20);
let copy = always_returns_no_lifetime(&t1);
t1 = Types::NoLifetime(30);
}
playground
The error is:
error[E0506]: cannot assign to `t1` because it is borrowed
--> src/main.rs:23:5
|
21 | let copy = always_returns_no_lifetime(&t1);
| -- borrow of `t1` occurs here
22 |
23 | t1 = Types::NoLifetime(30);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ assignment to borrowed `t1` occurs here
The return type of your function is wrong. If the return value is guaranteed not to have any lifetime, then it should be marked as such, and not tied down to an arbitrary lifetime:
fn always_returns_no_lifetime(...) -> Types<'static>;
With this change, you actually no longer need any input lifetime either, since they are only useful to tie the input and output, leading the following signature:
fn always_returns_no_lifetime(some_type: &Types) -> Types<'static>;
Unfortunately, this means that clone is now out of the table, as it clones the lifetime, so the implementation has to change too:
fn always_returns_no_lifetime(some_type: &Types) -> Types<'static> {
match *some_type {
Types::HasLifetime(_)
=> panic!("I only return values that have no lifetime"),
Types::NoLifetime(i) => Types::NoLifetime(i),
Types::AlsoNoLifetime(i) => Types::AlsoNoLifetime(i),
Types::AlsoAlsoNoLifetime(i) => Types::AlsoAlsoNoLifetime(i),
}
}
The benefit of this implementation can be demonstrated in the following example:
fn tie<'a>(text: &'a str) -> Types<'a> {
if text[0] == 'a' {
Types::HasLifetime(text)
} else {
Types::NoLifetime(0)
}
}
fn main() {
let no_lifetime = {
let string = String::from("Hello, world");
let has_lifetime = tie(&*string);
always_returns_no_lifetime(&has_lifetime)
};
// Requires deriving Debug, all structs really should...
println!("{:?}", no_lifetime);
}
If you preserve the lifetime when you don't need it, you cannot compile this example, it's an unnecessary restriction.
Instead of applying the 'a lifetime parameter on the reference, apply it on Types, as you already did with the return type. The lifetime on the reference is not important when you call .clone() on it.
fn always_returns_no_lifetime<'a>(some_type: &Types<'a>) -> Types<'a> {
match *some_type {
Types::HasLifetime(text) => panic!("I only return the type that has no lifetime"),
_ => some_type.clone()
}
}
Why does the call self.f2() in the following code trip the borrow checker? Isn't the else block in a different scope? This is quite a conundrum!
use std::str::Chars;
struct A;
impl A {
fn f2(&mut self) {}
fn f1(&mut self) -> Option<Chars> {
None
}
fn f3(&mut self) {
if let Some(x) = self.f1() {
} else {
self.f2()
}
}
}
fn main() {
let mut a = A;
}
Playground
error[E0499]: cannot borrow `*self` as mutable more than once at a time
--> src/main.rs:16:13
|
13 | if let Some(x) = self.f1() {
| ---- first mutable borrow occurs here
...
16 | self.f2()
| ^^^^ second mutable borrow occurs here
17 | }
| - first borrow ends here
Doesn't the scope of the borrow for self begin and end with the self.f1() call? Once the call from f1() has returned f1() is not using self anymore hence the borrow checker should not have any problem with the second borrow. Note the following code fails too...
// ...
if let Some(x) = self.f1() {
self.f2()
}
// ...
Playground
I think the second borrow should be fine here since f1 and f3 are not using self at the same time as f2.
I put together an example to show off the scoping rules here:
struct Foo {
a: i32,
}
impl Drop for Foo {
fn drop(&mut self) {
println!("Foo: {}", self.a);
}
}
fn generate_temporary(a: i32) -> Option<Foo> {
if a != 0 { Some(Foo { a: a }) } else { None }
}
fn main() {
{
println!("-- 0");
if let Some(foo) = generate_temporary(0) {
println!("Some Foo {}", foo.a);
} else {
println!("None");
}
println!("-- 1");
}
{
println!("-- 0");
if let Some(foo) = generate_temporary(1) {
println!("Some Foo {}", foo.a);
} else {
println!("None");
}
println!("-- 1");
}
{
println!("-- 0");
if let Some(Foo { a: 1 }) = generate_temporary(1) {
println!("Some Foo {}", 1);
} else {
println!("None");
}
println!("-- 1");
}
{
println!("-- 0");
if let Some(Foo { a: 2 }) = generate_temporary(1) {
println!("Some Foo {}", 1);
} else {
println!("None");
}
println!("-- 1");
}
}
This prints:
-- 0
None
-- 1
-- 0
Some Foo 1
Foo: 1
-- 1
-- 0
Some Foo 1
Foo: 1
-- 1
-- 0
None
Foo: 1
-- 1
In short, it seems that the expression in the if clause lives through both the if block and the else block.
On the one hand it is not surprising since it is indeed required to live longer than the if block, but on the other hand it does indeed prevent useful patterns.
If you prefer a visual explanation:
if let pattern = foo() {
if-block
} else {
else-block
}
desugars into:
{
let x = foo();
match x {
pattern => { if-block }
_ => { else-block }
}
}
while you would prefer that it desugars into:
bool bypass = true;
{
let x = foo();
match x {
pattern => { if-block }
_ => { bypass = false; }
}
}
if not bypass {
else-block
}
You are not the first one being tripped by this, so this may be addressed at some point, despite changing the meaning of some code (guards, in particular).
It's annoying, but you can work around this by introducing an inner scope and changing the control flow a bit:
fn f3(&mut self) {
{
if let Some(x) = self.f1() {
// ...
return;
}
}
self.f2()
}
As pointed out in the comments, this works without the extra braces. This is because an if or if...let expression has an implicit scope, and the borrow lasts for this scope:
fn f3(&mut self) {
if let Some(x) = self.f1() {
// ...
return;
}
self.f2()
}
Here's a log of an IRC chat between Sandeep Datta and mbrubeck:
mbrubeck: std:tr::Chars contains a borrowed reference to the string that created it. The full type name is Chars<'a>. So f1(&mut self) -> Option<Chars> without elision is f1(&'a mut self) -> Option<Chars<'a>> which means that self remains borrowed as long as
the return value from f1 is in scope.
Sandeep Datta: Can I use 'b for self and 'a for Chars to avoid this problem?
mbrubeck: Not if you are actually returning an iterator over something from self. Though if you can make a function from &self -> Chars (instead of &mut self -> Chars) that would fix the issue.
As of Rust 2018, available in Rust 1.31, the original code will work as-is. This is because Rust 2018 enables non-lexical lifetimes.
A mutable reference is a very strong guarantee: that there's only one pointer to a particular memory location. Since you've already had one &mut borrow, you can't also have a second. That would introduce a data race in a multithreaded context, and iterator invalidation and other similar issues in a single-threaded context.
Right now, borrows are based on lexical scope, and so the first borrow lasts until the end of the function, period. Eventually, we hope to relax this restriction, but it will take some work.
Here is how you can get rid of the spurious errors. I am new to Rust so there may be serious errors in the following explanation.
use std::str::Chars;
struct A<'a> {
chars: Chars<'a>,
}
The 'a here is a lifetime parameter (just like template parameters in C++). Types can be parameterised by lifetimes in Rust.
The Chars type also takes a lifetime parameter. What this implies is that the Chars type probably has a member element which needs a lifetime parameter. Lifetime parameters only make sense on references (since lifetime here actually means "lifetime of a borrow").
We know that Chars needs to keep a reference to the string from which it was created, 'a will probably be used to denote the source string's lifetime.
Here we simply supply 'a as the lifetime parameter to Chars telling the Rust compiler that the lifetime of Chars is the same as the lifetime of the struct A. IMO "lifetime 'a of type A" should be read as "lifetime 'a of the references contained in the struct A".
I think the struct implementation can be parameterised independently from the struct itself hence we need to repeat the parameters with the impl keyword. Here we bind the name 'a to the lifetime of the struct A.
impl<'a> A<'a> {
The name 'b is introduced in the context of the function f2. Here it is used to bind with the lifetime of the reference &mut self.
fn f2<'b>(&'b mut self) {}
The name 'b is introduced in the context of the function f1.This 'b does not have a direct relationship with the 'b introduced by f2 above.
Here it is used to bind with the lifetime of the reference &mut self. Needless to say this reference also does not have any relationship with the &mut self in the previous function, this is a new independent borrow of self.
Had we not used explicit lifetime annotation here Rust would have used its lifetime elision rules to arrive at the following function signature...
//fn f1<'a>(&'a mut self) -> Option<Chars<'a>>
As you can see this binds the lifetime of the reference &mut self parameter to the lifetime of the Chars object being returned from this function (this Chars object need not be the same as self.chars) this is absurd since the returned Chars will outlive the &mut self reference. Hence we need to separate the two lifetimes as follows...
fn f1<'b>(&'b mut self) -> Option<Chars<'a>> {
self.chars.next();
Remember &mut self is a borrow of self and anything referred to by &mut self is also a borrow. Hence we cannot return Some(self.chars) here. self.chars is not ours to give (Error: cannot move out of borrowed content.).
We need to create a clone of self.chars so that it can be given out.
Some(self.chars.clone())
Note here the returned Chars has the same lifetime as the struct A.
And now here is f3 unchanged and without compilation errors!
fn f3<'b>(&'b mut self) {
if let Some(x) = self.f1() { //This is ok now
} else {
self.f2() //This is also ok now
}
}
The main function just for completeness...
fn main() {
let mut a = A { chars:"abc".chars() };
a.f3();
for c in a.chars {
print!("{}", c);
}
}
I have updated the code the make the lifetime relationships clearer.
I'm writing a parser generator as a project to learn rust, and I'm running into something I can't figure out with lifetimes and closures. Here's my simplified case (sorry it's as complex as it is, but I need to have the custom iterator in the real version and it seems to make a difference in the compiler's behavior):
Playpen link: http://is.gd/rRm2aa
struct MyIter<'stat, T:Iterator<&'stat str>>{
source: T
}
impl<'stat, T:Iterator<&'stat str>> Iterator<&'stat str> for MyIter<'stat, T>{
fn next(&mut self) -> Option<&'stat str>{
self.source.next()
}
}
struct Scanner<'stat,T:Iterator<&'stat str>>{
input: T
}
impl<'main> Scanner<'main, MyIter<'main,::std::str::Graphemes<'main>>>{
fn scan_literal(&'main mut self) -> Option<String>{
let mut token = String::from_str("");
fn get_chunk<'scan_literal,'main>(result:&'scan_literal mut String,
input: &'main mut MyIter<'main,::std::str::Graphemes<'main>>)
-> Option<&'scan_literal mut String>{
Some(input.take_while(|&chr| chr != "\"")
.fold(result, |&mut acc, chr|{
acc.push_str(chr);
&mut acc
}))
}
get_chunk(&mut token,&mut self.input);
println!("token is {}", token);
Some(token)
}
}
fn main(){
let mut scanner = Scanner{input:MyIter{source:"\"foo\"".graphemes(true)}};
scanner.scan_literal();
}
There are two problems I know of here. First, I have to shadow the 'main lifetime in the get_chunk function (I tried using the one in the impl, but the compiler complains that 'main is undefined inside get_chunk). I think it will still work out because the call to get_chunk later will match the 'main from the impl with the 'main from get_chunk, but I'm not sure that's right.
The second problem is that the &mut acc inside the closure needs to have a lifetime of 'scan_literal in order to work like I want it to (accumulating characters until the first " is encountered for this example). I can't add an explicit lifetime to &mut acc though, and the compiler says its lifetime is limited to the closure itself, and thus I can't return the reference to use in the next iteration of fold. I've gotten the function to compile and run in various other ways, but I don't understand what the problem is here.
My main question is: Is there any way to explicitly specify the lifetime of an argument to a closure? If not, is there a better way to accumulate the string using fold without doing multiple copies?
First, about lifetimes. Functions defined inside other functions are static, they are not connected with their outside code in any way. Consequently, their lifetime parameters are completely independent. You don't want to use 'main as a lifetime parameter for get_chunk() because it will shadow the outer 'main lifetime and give nothing but confusion.
Next, about closures. This expression:
|&mut acc, chr| ...
very likely does not what you really think it does. Closure/function arguments allow irrefutable patterns in them, and & have special meaning in patterns. Namely, it dereferences the value it is matched against, and assigns its identifier to this dereferenced value:
let x: int = 10i;
let p: &int = &x;
match p {
&y => println!("{}", y) // prints 10
}
You can think of & in a pattern as an opposite to & in an expression: in an expression it means "take a reference", in a pattern it means "remove the reference".
mut, however, does not belong to & in patterns; it belongs to the identifier and means that the variable with this identifier is mutable, i.e. you should write not
|&mut acc, chr| ...
but
|& mut acc, chr| ...
You may be interested in this RFC which is exactly about this quirk in the language syntax.
It looks like that you want to do a very strange thing, I'm not sure I understand where you're getting at. It is very likely that you are confusing different string kinds. First of all, you should read the official guide which explains ownership and borrowing and when to use them (you may also want to read the unfinished ownership guide; it will soon get into the main documentation tree), and then you should read strings guide.
Anyway, your problem can be solved in much simpler and generic way:
#[deriving(Clone)]
struct MyIter<'s, T: Iterator<&'s str>> {
source: T
}
impl<'s, T: Iterator<&'s str>> Iterator<&'s str> for MyIter<'s, T>{
fn next(&mut self) -> Option<&'s str>{ // '
self.source.next()
}
}
#[deriving(Clone)]
struct Scanner<'s, T: Iterator<&'s str>> {
input: T
}
impl<'m, T: Iterator<&'m str>> Scanner<'m, T> { // '
fn scan_literal(&mut self) -> Option<String>{
fn get_chunk<'a, T: Iterator<&'a str>>(input: T) -> Option<String> {
Some(
input.take_while(|&chr| chr != "\"")
.fold(String::new(), |mut acc, chr| {
acc.push_str(chr);
acc
})
)
}
let token = get_chunk(self.input.by_ref());
println!("token is {}", token);
token
}
}
fn main(){
let mut scanner = Scanner{
input: MyIter {
source: "\"foo\"".graphemes(true)
}
};
scanner.scan_literal();
}
You don't need to pass external references into the closure; you can generate a String directly in fold() operation. I also generified your code and made it more idiomatic.
Note that now impl for Scanner also works with arbitrary iterators returning &str. It is very likely that you want to write this instead of specializing Scanner to work only with MyIter with Graphemes inside it. by_ref() operation turns &mut I where I is an Iterator<T> into J, where J is an Iterator<T>. It allows further chaining of iterators even if you only have a mutable reference to the original iterator.
By the way, your code is also incomplete; it will only return Some("") because the take_while() will stop at the first quote and won't scan further. You should rewrite it to take initial quote into account.