According to The Rust Programming Language, ch15-03, std::mem::drop takes an object, receives its ownership, and calls its drop function.
That's what this code does:
fn my_drop<T>(x: T) {}
fn main() {
let x = 5;
let y = &x;
let mut z = 4;
let v = vec![3, 4, 2, 5, 3, 5];
my_drop(v);
}
Is this what std::mem::drop does? Does it perform any other cleanup tasks other than these?
Let's take a look at the source:
#[inline]
#[stable(feature = "rust1", since = "1.0.0")]
pub fn drop<T>(_x: T) { }
#[inline] gives a hint to the compiler that the function should be inlined. #[stable] is used by the standard library to mark APIs that are available on the stable channel. Otherwise, it's really just an empty function! When _x goes out of scope as drop returns, its destructor is run; there is no other way to perform cleanup tasks implicitly in Rust.
Related
I am trying to create a function in Rust which will sample from M normal distributions N times. I have the sequential version below, which runs fine. I am trying to parallelize it using Rayon, but am encountering the error
Rc<UnsafeCell<ReseedingRng<rand_chacha::chacha::ChaCha12Core, OsRng>>> cannot be sent between threads safely
It seems my rand::thread_rng does not implement the traits Send and Sync. I tried using StdRng and OsRng which both do, to no avail because then I get errors that the variables pred and rng cannot be borrowed as mutable because they are captured in a Fn closure.
This is the working code below. It errors when I change the first into_iter() to into_par_iter().
use rand_distr::{Normal, Distribution};
use std::time::Instant;
use rayon::prelude::*;
fn rprednorm(n: i32, means: Vec<f64>, sds: Vec<f64>) -> Vec<Vec<f64>> {
let mut rng = rand::thread_rng();
let mut preds = vec![vec![0.0; n as usize]; means.len()];
(0..means.len()).into_iter().for_each(|i| {
(0..n).into_iter().for_each(|j| {
let normal = Normal::new(means[i], sds[i]).unwrap();
preds[i][j as usize] = normal.sample(&mut rng);
})
});
preds
}
fn main() {
let means = vec![0.0; 67000];
let sds = vec![1.0; 67000];
let start = Instant::now();
let preds = rprednorm(100, means, sds);
let duration = start.elapsed();
println!("{:?}", duration);
}
Any advice on how to make these two iterators parallel?
Thanks.
It seems my rand::thread_rng does not implement the traits Send and Sync.
Why are you trying to send a thread_rng? The entire point of thread_rng is that it's a per-thread RNG.
then I get errors that the variables pred and rng cannot be borrowed as mutable because they are captured in a Fn closure.
Well yes, you need to Clone the StdRng (or Copy the OsRng) into each closure. As for pred, that can't work for a similar reason: once you parallelise the loop the compiler does not know that every i is distinct, so as far as it's concerned the write access to i could overlap (you could have two iterations running in parallel which try to write to the same place at the same time) which is illegal.
The solution is to use rayon to iterate in parallel over the destination vector:
fn rprednorm(n: i32, means: Vec<f64>, sds: Vec<f64>) -> Vec<Vec<f64>> {
let mut preds = vec![vec![0.0; n as usize]; means.len()];
preds.par_iter_mut().enumerate().for_each(|(i, e)| {
let mut rng = rand::thread_rng();
(0..n).into_iter().for_each(|j| {
let normal = Normal::new(means[i], sds[i]).unwrap();
e[j as usize] = normal.sample(&mut rng);
})
});
preds
}
Alternatively with OsRng, it's just a marker ZST, so you can refer to it as a value:
fn rprednorm(n: i32, means: Vec<f64>, sds: Vec<f64>) -> Vec<Vec<f64>> {
let mut preds = vec![vec![0.0; n as usize]; means.len()];
preds.par_iter_mut().enumerate().for_each(|(i, e)| {
(0..n).into_iter().for_each(|j| {
let normal = Normal::new(means[i], sds[i]).unwrap();
e[j as usize] = normal.sample(&mut rand::rngs::OsRng);
})
});
preds
}
StdRng doesn't seem very suitable to this use-case, as you'll either have to create one per toplevel iteration to get different samplings, or you'll have to initialise a base rng then clone it once per spark, and they'll all have the same sequence (as they'll share a seed).
I want to modify a collection in place before returning it:
fn main() {
println!("{:?}", compute()); // should print [[2, 1, 0], [5, 4, 3]]
}
// u8 is just a placeholder, so impl Copy is considered cheating :)
fn compute() -> Vec<Vec<u8>> {
let a = vec![0, 1, 2];
let b = vec![3, 4, 5];
let mut result = Vec::new();
result.push(a);
result.push(b);
// avoids allocations from:
//
// result.iter()
// .map(|r| {
// r.reverse()
// r
// })
// .collect::<Vec<_>>()
result.into_iter().for_each(|mut r| r.reverse());
// errors out: the collection was consumed the line above
result
}
A collection was already allocated with Vec::new(), so allocating a second collection here seems like a waste. I am assuming that's what .collect() does.
How do I avoid the allocation in excess?
Is there any easy way to know how many allocations are happening? In golang it was as easy as go test -bench=., but I can't find anything similar when it comes to Rust.
Link to playground
You need to use a &mut to each of the inside vectors, for that you can just use iter_mut which uses &mut Self instead of Self for the outer vector.
// u8 is just a placeholder, so impl Copy is considered cheating :)
fn compute() -> Vec<Vec<u8>> {
let a = vec![0, 1, 2];
let b = vec![3, 4, 5];
let mut result = Vec::new();
result.push(a);
result.push(b);
result.iter_mut().for_each(|r| r.reverse());
result
}
Playground
I'm using filter, but I don't understand why I should use **x > 1 for a slice but use *x > 1 for a range.
fn main() {
let a = [0, 1, 2, 3];
let a_iter = a.iter().filter(|x: &&i32| **x > 1); // x: &&i32
let x: Vec<&i32> = a_iter.collect();
println!("{:?}", x);
let b = 0..4;
let b_iter = b.filter(|x: &i32| *x > 1); // x: &i32
let y: Vec<i32> = b_iter.collect();
println!("{:?}", y);
}
The docs say it should be **x > 1.
slice::iter, such as a.iter() in your example, produces an iterator over references to values. Ranges are iterators that produce non-reference values.
The filter(<closure>) method takes a <closure> that takes iterator values by reference, so if your iterator already produces references you'll get a reference to a reference, and if your iterator produces non-reference values then you'll get references to those values.
The difference becomes easier to understand if we use a Vec for both examples:
fn main() {
let a = vec![0, 1, 2, 3];
let a_iter = a.iter(); // iter() returns iterator over references
let x: Vec<&i32> = a_iter.filter(|x: &&i32| **x > 1).collect();
println!("{:?}", x);
let b = vec![0, 1, 2, 3];
let b_iter = a.into_iter(); // into_iter() returns iterator over values
let y: Vec<i32> = b_iter.filter(|x: &i32| *x > 1).collect();
println!("{:?}", y);
}
playground
This is because in the array-example, you first call .iter() to create an iterator, which borrows the array and hence to values in it; the closure-argument to filter receives a borrowed version of the iterator's Item, so it's borrowed again, which makes it a &&i32.
In the Range-case, you call filter directly, as a Range is an iterator. The iterator, therefore, contains owned values and the closure in filter borrows that, which makes its type &i32.
You can see that if you try to let y = b; after the let b_iter = ...-line: You'll get a use of moved value-error because b was consumed by the iterator you used in b.filter
An example from the documentation about vectors:
let v = vec![1, 2, 3, 4, 5];
let third: &i32 = &v[2];
println!("The third element is {}", third);
match v.get(2) {
Some(third) => println!("The third element is {}", third),
None => println!("There is no third element."),
}
I can't see why third needs to be a reference. let third: i32 = v[2] seems to work just as well. What does making it a reference achieve?
Similarly:
let v = vec![100, 32, 57];
for i in &v {
println!("{}", i);
}
why is it in &v instead of just in v?
let third: i32 = v[2] works because i32 implements Copy trait. They don't get moved out when indexing the vector, they get copied instead.
When you have a vector of non Copy type, it is a different story.
let v = vec![
"1".to_string(),
"2".to_string(),
"3".to_string(),
"4".to_string(),
"5".to_string(),
];
let third = &v[2]; // This works
// let third = v[2]; // This doesn't work because String doesn't implement Copy
As for the second question about the loop, for loop is syntactic sugar for IntoIterator which moves and consumes.
So, when you need to use v after the loop, you don’t want to move it. You want to borrow it with &v or v.iter() instead.
let v = vec![100, 32, 57];
for i in &v { // borrow, not move
println!("{}", i);
}
println!("{}", v[0]); // if v is moved above, this doesn't work
I'm coming from a C (and to a lesser extent, C++) background. I wrote the following code snippet:
fn main() {
let my_array = [1, 2, 3];
let print_me = |j| println!("= {}", j);
for k in my_array.iter() {
print_me(k);
}
}
This compiled and ran as expected, but then I specified the type of the argument passed to the closure print_me thus:
fn main() {
let my_array = [1, 2, 3];
let print_me = |j: i32| println!("= {}", j);
for k in my_array.iter() {
print_me(k);
}
}
I got a compilation error:
error[E0308]: mismatched types
--> src/main.rs:6:22
|
6 | print_me(k);
| ^
| |
| expected i32, found &{integer}
| help: consider dereferencing the borrow: `*k`
|
= note: expected type `i32`
found type `&{integer}`
Now this confused me until I changed k to &k in the for statement, which worked fine:
fn main() {
let my_array = [1, 2, 3];
let print_me = |j: i32| println!("= {}", j);
for &k in my_array.iter() {
print_me(k);
}
}
It seems that I misunderstood the for syntax itself -- or maybe the exact workings of an iterator -- or maybe the usage syntax of a reference vis-a-vis a pointer [which are related but distinct in C++].
In the construct for A in B { C1; C2; ... Cn }, what exactly are A and B supposed to be?
First of all, here's a link to the definition of for in the reference.
To summarise, B is any expression which evaluates to something that can be converted into a value that implements the Iterator<T> trait, whilst A is a irrefutable pattern that binds values of type T.
In your specific case, slice::iter returns an Iter<i32>, which implements Iterator<Item = &i32>. That is, it doesn't yield i32s, it yields &i32s.
Thus, in both the first and second examples, k is actually binding to &i32s, not i32s. When you specified the type of the closure, you were actually specifying the wrong type. The reason the final example works is because A is a pattern, not a variable name. What &k is actually doing is "de-structuring" the &i32, binding the i32 part to a variable named k.
The "irrefutable" part simply means that the pattern must always work. For example, you can't do for Some(x) in thingy where thingy implements Iterator<Option<_>>; Some(x) would not necessarily be valid for every element in the iterator; thus, it's a refutable pattern.
Many iterators actually return a reference rather than a value. To be sure, you have to check the return type of .iter(), which should be of the form Iterator<Item = X>: X will be the type of the variable returned.
So here:
fn main() {
let my_array = [1, 2, 3];
let print_me = |j: i32| println!("= {}", j);
for k in my_array.iter() {
print_me(k);
}
}
This X is &i32 (a reference to i32), and therefore k has type &i32.
This is why, when calling print_me, there is an error: &i32 is passed where i32 is expected.
There are multiple possible fixes here:
specify a different type to print_me:
let print_me = |j: &i32| println!("= {}", j);
dereference the value of k:
print_me(*k);
change the type of k by destructuring in the loop:
for &k in my_array.iter() { ... }
The destructuring occurs because for .. in accepts an irrefutable pattern, so you can pattern match like you would do in a match expression, except that the variable's type has to match (otherwise you get a compiler time error).
To better illustrate it, we can use a slightly more complicated example:
fn main() {
let my_array = [(1, 2), (2, 3), (3, 4)];
let print_me = |a: i32, b: i32| println!("= {} {}", a, b);
for &(j, k) in my_array.iter() {
print_me(j, k)
}
}
The type of my_array is [(i32, i32)]: an array of tuples of 2 i32. The result of .iter() is therefore of type Iterator<Item = &(i32, i32)>: an iterator to a reference to a tuple of 2 i32 aka &(i32, i32).
When we use the irrefutable pattern &(j, k) what happens is that we destructure the tuple so that:
the first element binds to j (inferred to be of type i32, only works because i32 is Copy)
the second element binds to k ((inferred to be of type i32)
j and k thus become temporary copies of the i32 inside this element.