Given the following simplified program:
#[macro_use] extern crate log;
extern crate ansi_term;
extern crate fern;
extern crate time;
extern crate threadpool;
extern crate id3;
mod logging;
use std::process::{exit, };
use ansi_term::Colour::{Yellow, Green};
use threadpool::ThreadPool;
use std::sync::mpsc::channel;
use std::path::{Path};
use id3::Tag;
fn main() {
logging::setup_logging();
let n_jobs = 2;
let files = vec!(
"/tmp/The Dynamics - Version Excursions/01-13- Move on Up.mp3",
"/tmp/The Dynamics - Version Excursions/01-09- Whole Lotta Love.mp3",
"/tmp/The Dynamics - Version Excursions/01-10- Feel Like Making Love.mp3"
);
let pool = ThreadPool::new(n_jobs);
let (tx, rx) = channel();
let mut counter = 0;
for file_ in files {
let file_ = Path::new(file_);
counter = counter + 1;
let tx = tx.clone();
pool.execute(move || {
debug!("sending {} from thread", Yellow.paint(counter.to_string()));
let tag = Tag::read_from_path(file_).unwrap();
let a_name = tag.artist().unwrap();
debug!("recursed file from: {} {}",
Green.paint(a_name), file_.display());
tx.send(".").unwrap();
// TODO amb: not working..
// tx.send(a_name).unwrap();
});
}
for value in rx.iter().take(counter) {
debug!("receiving {} from thread", Green.paint(value));
}
exit(0);
}
Everything works as expected, unless the one commented line (tx.send(a_name).unwrap();) is put back in. In that case I get the following error:
error: `tag` does not live long enough
let a_name = tag.artist().unwrap();
^~~
note: reference must be valid for the static lifetime...
note: ...but borrowed value is only valid for the block suffix following statement 1 at 39:58
let tag = Tag::read_from_path(file_).unwrap();
let a_name = tag.artist().unwrap();
debug!("recursed file from: {} {}",
Green.paint(a_name), file_.display());
...
Generally I understand what the compiler tells me, but I don't see a problem since the variable tag is defined inside of the closure block. The only problem that I can guess is, that the variable tx is cloned outside and therefore can collide with the lifetime of tag.
My goal is to put all the current logic in the thread-closure inside of the thread, since this is the "processing" I want to spread to multiple threads. How can I accomplish this, but still send some value to the longer existing tx?
I'm using the following Rust version:
$ rustc --version
rustc 1.9.0 (e4e8b6668 2016-05-18)
$ cargo --version
cargo 0.10.0-nightly (10ddd7d 2016-04-08)
a_name is &str borrowed from tag. Its lifetime is therefore bounded by tag. Sending non 'static references down a channel to another thread is unsafe. It refers to something on the threads stack which might not even exist anymore once the receiver tries to access it.
In your case you should promote a_name to an owned value of type String, which will be moved to the receiver thread.
tx.send(a_name.to_owned()).unwrap();
Related
I saw in the Rust book that you can define two different variables with the same name:
let hello = "Hello";
let hello = "Goodbye";
println!("My variable hello contains: {}", hello);
This prints out:
My variable hello contains: Goodbye
What happens with the first hello? Does it get freed up? How could I access it?
I know it would be bad to name two variables the same, but if this happens by accident because I declare it 100 lines below it could be a real pain.
Rust does not have a garbage collector.
Does Rust free up the memory of overwritten variables?
Yes, otherwise it'd be a memory leak, which would be a pretty terrible design decision. The memory is freed when the variable is reassigned:
struct Noisy;
impl Drop for Noisy {
fn drop(&mut self) {
eprintln!("Dropped")
}
}
fn main() {
eprintln!("0");
let mut thing = Noisy;
eprintln!("1");
thing = Noisy;
eprintln!("2");
}
0
1
Dropped
2
Dropped
what happens with the first hello
It is shadowed.
Nothing "special" happens to the data referenced by the variable, other than the fact that you can no longer access it. It is still dropped when the variable goes out of scope:
struct Noisy;
impl Drop for Noisy {
fn drop(&mut self) {
eprintln!("Dropped")
}
}
fn main() {
eprintln!("0");
let thing = Noisy;
eprintln!("1");
let thing = Noisy;
eprintln!("2");
}
0
1
2
Dropped
Dropped
See also:
Is the resource of a shadowed variable binding freed immediately?
I know it would be bad to name two variables the same
It's not "bad", it's a design decision. I would say that using shadowing like this is a bad idea:
let x = "Anna";
println!("User's name is {}", x);
let x = 42;
println!("The tax rate is {}", x);
Using shadowing like this is reasonable to me:
let name = String::from(" Vivian ");
let name = name.trim();
println!("User's name is {}", name);
See also:
Why do I need rebinding/shadowing when I can have mutable variable binding?
but if this happens by accident because I declare it 100 lines below it could be a real pain.
Don't have functions that are so big that you "accidentally" do something. That's applicable in any programming language.
Is there a way of cleaning memory manually?
You can call drop:
eprintln!("0");
let thing = Noisy;
drop(thing);
eprintln!("1");
let thing = Noisy;
eprintln!("2");
0
Dropped
1
2
Dropped
However, as oli_obk - ker points out, the stack memory taken by the variable will not be freed until the function exits, only the resources taken by the variable.
All discussions of drop require showing its (very complicated) implementation:
fn drop<T>(_: T) {}
What if I declare the variable in a global scope outside of the other functions?
Global variables are never freed, if you can even create them to start with.
There is a difference between shadowing and reassigning (overwriting) a variable when it comes to drop order.
All local variables are normally dropped when they go out of scope, in reverse order of declaration (see The Rust Programming Language's chapter on Drop). This includes shadowed variables. It's easy to check this by wrapping the value in a simple wrapper struct that prints something when it (the wrapper) is dropped (just before the value itself is dropped):
use std::fmt::Debug;
struct NoisyDrop<T: Debug>(T);
impl<T: Debug> Drop for NoisyDrop<T> {
fn drop(&mut self) {
println!("dropping {:?}", self.0);
}
}
fn main() {
let hello = NoisyDrop("Hello");
let hello = NoisyDrop("Goodbye");
println!("My variable hello contains: {}", hello.0);
}
prints the following (playground):
My variable hello contains: Goodbye
dropping "Goodbye"
dropping "Hello"
That's because a new let binding in a scope does not overwrite the previous binding, so it's just as if you had written
let hello1 = NoisyDrop("Hello");
let hello2 = NoisyDrop("Goodbye");
println!("My variable hello contains: {}", hello2.0);
Notice that this behavior is different from the following, superficially very similar, code (playground):
fn main() {
let mut hello = NoisyDrop("Hello");
hello = NoisyDrop("Goodbye");
println!("My variable hello contains: {}", hello.0);
}
which not only drops them in the opposite order, but drops the first value before printing the message! That's because when you assign to a variable (instead of shadowing it with a new one), the original value gets dropped first, before the new value is moved in.
I began by saying that local variables are "normally" dropped when they go out of scope. Because you can move values into and out of variables, the analysis of figuring out when variables need to be dropped can sometimes not be done until runtime. In such cases, the compiler actually inserts code to track "liveness" and drop those values when necessary, so you can't accidentally cause leaks by overwriting a value. (However, it's still possible to safely leak memory by calling mem::forget, or by creating an Rc-cycle with internal mutability.)
See also
What's the semantic of assignment in Rust?
There are a few things to note here:
In the program you gave, when compiling it, the "Hello" string does not appear in the binary. This might be a compiler optimization because the first value is not used.
fn main(){
let hello = "Hello xxxxxxxxxxxxxxxx"; // Added for searching more easily.
let hello = "Goodbye";
println!("My variable hello contains: {}", hello);
}
Then test:
$ rustc ./stackoverflow.rs
$ cat stackoverflow | grep "xxx"
# No results
$ cat stackoverflow | grep "Goodbye"
Binary file (standard input) matches
$ cat stackoverflow | grep "My variable hello contains"
Binary file (standard input) matches
Note that if you print the first value, the string does appear in the binary though, so this proves that this is a compiler optimization to not store unused values.
Another thing to consider is that both values assigned to hello (i.e. "Hello" and "Goodbye") have a &str type. This is a pointer to a string stored statically in the binary after compiling. An example of a dynamically generated string would be when you generate a hash from some data, like MD5 or SHA algorithms (the resulting string does not exist statically in the binary).
fn main(){
// Added the type to make it more clear.
let hello: &str = "Hello";
let hello: &str = "Goodbye";
// This is wrong (does not compile):
// let hello: String = "Goodbye";
println!("My variable hello contains: {}", hello);
}
This means that the variable is simply pointing to a location in the static memory. No memory gets allocated during runtime, nor gets freed. Even if the optimization mentioned above didn't exist (i.e. omit unused strings), only the memory address location pointed by hello would change, but the memory is still used by static strings.
The story would be different for a String type, and for that refer to the other answers.
I am trying to fast forward time to do some tests for a custom runtime module. I have looked at the answer from this thread and followed the answer to use Timestamp, however, I am unable to access the set_timestamp method.
setup:
#[cfg(test)]
mod tests {
use super::*;
use support::dispatch::Vec;
use runtime_primitives::traits::{Hash};
use runtime_io::with_externalities;
use primitives::{H256, Blake2Hasher};
use timestamp;
use support::{impl_outer_origin, assert_ok, assert_noop};
use runtime_primitives::{
BuildStorage,
traits::{BlakeTwo256, IdentityLookup},
testing::{Digest, DigestItem, Header}
};
impl_outer_origin! {
pub enum Origin for Test {}
}
#[derive(Clone, Eq, PartialEq)]
pub struct Test;
impl system::Trait for Test {
type Origin = Origin;
type Index = u64;
type BlockNumber = u64;
type Hash = H256;
type Hashing = BlakeTwo256;
type Digest = Digest;
type AccountId = u64;
type Lookup = IdentityLookup<Self::AccountId>;
type Header = Header;
type Event = ();
type Log = DigestItem;
}
impl super::Trait for Test {
type Event = ();
}
impl timestamp::Trait for Test {
type Moment = u64;
type OnTimestampSet = ();
}
type Pizza = Module<Test>;
And the error is below:
error[E0599]: no function or associated item named `set_timestamp` found for type
`srml_timestamp::Module<tests::Test>` in the current scope
|
254 | let now = <timestamp::Module<tests::Test>>::set_timestamp(9);
| ^^^^^^^^^^^^^ function or associated item
not found in `srml_timestamp::Module<tests::Test>`
In Substrate v1.0, the set_timestamp function has a #[cfg(feature = "std")] attribute on it:
https://github.com/paritytech/substrate/blob/v1.0/srml/timestamp/src/lib.rs#L276
This means it will only be visible if you are compiling with std. When you write tests, this should work, but I assume that this issue is appearing because you are trying to call it from within the runtime environment, which much be no_std.
If for some reason you do need to modify the timestamp from within your runtime, you should be able to do so directly:
https://github.com/paritytech/substrate/blob/v1.0/srml/timestamp/src/lib.rs#L249
<timestamp::Module<T>>::Now::put(new_time)
(I haven't tested this, but something like it should work).
Let me know if this helps.
In Substrate v1.0 you can declare
type Moment = timestamp::Module<Test>;
Then use it to set a specific timestamp.
Moment::set_timestamp(9);
If you want to get the timestamp value, you can do:
let now_timestamp = Moment::now();
I saw in the Rust book that you can define two different variables with the same name:
let hello = "Hello";
let hello = "Goodbye";
println!("My variable hello contains: {}", hello);
This prints out:
My variable hello contains: Goodbye
What happens with the first hello? Does it get freed up? How could I access it?
I know it would be bad to name two variables the same, but if this happens by accident because I declare it 100 lines below it could be a real pain.
Rust does not have a garbage collector.
Does Rust free up the memory of overwritten variables?
Yes, otherwise it'd be a memory leak, which would be a pretty terrible design decision. The memory is freed when the variable is reassigned:
struct Noisy;
impl Drop for Noisy {
fn drop(&mut self) {
eprintln!("Dropped")
}
}
fn main() {
eprintln!("0");
let mut thing = Noisy;
eprintln!("1");
thing = Noisy;
eprintln!("2");
}
0
1
Dropped
2
Dropped
what happens with the first hello
It is shadowed.
Nothing "special" happens to the data referenced by the variable, other than the fact that you can no longer access it. It is still dropped when the variable goes out of scope:
struct Noisy;
impl Drop for Noisy {
fn drop(&mut self) {
eprintln!("Dropped")
}
}
fn main() {
eprintln!("0");
let thing = Noisy;
eprintln!("1");
let thing = Noisy;
eprintln!("2");
}
0
1
2
Dropped
Dropped
See also:
Is the resource of a shadowed variable binding freed immediately?
I know it would be bad to name two variables the same
It's not "bad", it's a design decision. I would say that using shadowing like this is a bad idea:
let x = "Anna";
println!("User's name is {}", x);
let x = 42;
println!("The tax rate is {}", x);
Using shadowing like this is reasonable to me:
let name = String::from(" Vivian ");
let name = name.trim();
println!("User's name is {}", name);
See also:
Why do I need rebinding/shadowing when I can have mutable variable binding?
but if this happens by accident because I declare it 100 lines below it could be a real pain.
Don't have functions that are so big that you "accidentally" do something. That's applicable in any programming language.
Is there a way of cleaning memory manually?
You can call drop:
eprintln!("0");
let thing = Noisy;
drop(thing);
eprintln!("1");
let thing = Noisy;
eprintln!("2");
0
Dropped
1
2
Dropped
However, as oli_obk - ker points out, the stack memory taken by the variable will not be freed until the function exits, only the resources taken by the variable.
All discussions of drop require showing its (very complicated) implementation:
fn drop<T>(_: T) {}
What if I declare the variable in a global scope outside of the other functions?
Global variables are never freed, if you can even create them to start with.
There is a difference between shadowing and reassigning (overwriting) a variable when it comes to drop order.
All local variables are normally dropped when they go out of scope, in reverse order of declaration (see The Rust Programming Language's chapter on Drop). This includes shadowed variables. It's easy to check this by wrapping the value in a simple wrapper struct that prints something when it (the wrapper) is dropped (just before the value itself is dropped):
use std::fmt::Debug;
struct NoisyDrop<T: Debug>(T);
impl<T: Debug> Drop for NoisyDrop<T> {
fn drop(&mut self) {
println!("dropping {:?}", self.0);
}
}
fn main() {
let hello = NoisyDrop("Hello");
let hello = NoisyDrop("Goodbye");
println!("My variable hello contains: {}", hello.0);
}
prints the following (playground):
My variable hello contains: Goodbye
dropping "Goodbye"
dropping "Hello"
That's because a new let binding in a scope does not overwrite the previous binding, so it's just as if you had written
let hello1 = NoisyDrop("Hello");
let hello2 = NoisyDrop("Goodbye");
println!("My variable hello contains: {}", hello2.0);
Notice that this behavior is different from the following, superficially very similar, code (playground):
fn main() {
let mut hello = NoisyDrop("Hello");
hello = NoisyDrop("Goodbye");
println!("My variable hello contains: {}", hello.0);
}
which not only drops them in the opposite order, but drops the first value before printing the message! That's because when you assign to a variable (instead of shadowing it with a new one), the original value gets dropped first, before the new value is moved in.
I began by saying that local variables are "normally" dropped when they go out of scope. Because you can move values into and out of variables, the analysis of figuring out when variables need to be dropped can sometimes not be done until runtime. In such cases, the compiler actually inserts code to track "liveness" and drop those values when necessary, so you can't accidentally cause leaks by overwriting a value. (However, it's still possible to safely leak memory by calling mem::forget, or by creating an Rc-cycle with internal mutability.)
See also
What's the semantic of assignment in Rust?
There are a few things to note here:
In the program you gave, when compiling it, the "Hello" string does not appear in the binary. This might be a compiler optimization because the first value is not used.
fn main(){
let hello = "Hello xxxxxxxxxxxxxxxx"; // Added for searching more easily.
let hello = "Goodbye";
println!("My variable hello contains: {}", hello);
}
Then test:
$ rustc ./stackoverflow.rs
$ cat stackoverflow | grep "xxx"
# No results
$ cat stackoverflow | grep "Goodbye"
Binary file (standard input) matches
$ cat stackoverflow | grep "My variable hello contains"
Binary file (standard input) matches
Note that if you print the first value, the string does appear in the binary though, so this proves that this is a compiler optimization to not store unused values.
Another thing to consider is that both values assigned to hello (i.e. "Hello" and "Goodbye") have a &str type. This is a pointer to a string stored statically in the binary after compiling. An example of a dynamically generated string would be when you generate a hash from some data, like MD5 or SHA algorithms (the resulting string does not exist statically in the binary).
fn main(){
// Added the type to make it more clear.
let hello: &str = "Hello";
let hello: &str = "Goodbye";
// This is wrong (does not compile):
// let hello: String = "Goodbye";
println!("My variable hello contains: {}", hello);
}
This means that the variable is simply pointing to a location in the static memory. No memory gets allocated during runtime, nor gets freed. Even if the optimization mentioned above didn't exist (i.e. omit unused strings), only the memory address location pointed by hello would change, but the memory is still used by static strings.
The story would be different for a String type, and for that refer to the other answers.
Basically, I am looking for something more or less equivalent to the following C code:
int theGlobalCount = 0;
int
theGlobalCount_get() { return theGlobalCount; }
void
theGlobalCount_set(int n) { theGlobalCount = n; return; }
You could use a neat trick: declare a mutable global variable, and make a ref (aka mutable reference) point to it (no GC is required to make this work!). Then, implement functions to provide access to the mutable reference.
local
var theGlobalCount_var : int = 0
val theGlobalCount = ref_make_viewptr (view# theGlobalCount_var | addr# theGlobalCount_var)
in // in of [local]
fun
theGlobalCount_get () : int = ref_get_elt (theGlobalCount)
fun
theGlobalCount_set (n: int): void = ref_set_elt (theGlobalCount, n)
end // end of [local]
Note that declarations inside local-in are visible only to code inside in-end. Therefore, neither theGlobalCount_var nor theGlobalCount are visible outside the scope of the local.
Full code: glot.io
You can also use the extvar feature to update an external global variable (declared in the target language). This is very useful if you compile ATS to a language that does not support explicit pointers (e.g., JavaScript). Here is a running example that makes use of this feature:
http://www.ats-lang.org/SERVER/MYCODE/Patsoptaas_serve.php?mycode_url=http://pastebin.com/raw/MsXhVE0A
I get a rust compiler error:
src/main.rs:33:31: 33:35 error: can't capture dynamic environment in a fn item; use the || { ... } closure form instead
The error occurs because I have a function in which I declare a variable using let, and then have an inner function in which I try to use this variable that is closed over.
I am fairly certain that this is a beginner question, so sorry in advance if this has a very straight forward answer!
Note that I am using this inner function as a callback somewhere, and thus using a closure, like
let closure_fn = | args | -> () { do stuff };
... is not going to be an appropriate solution for me.
extern crate nickel;
use std::io::net::ip::Ipv4Addr;
use nickel::{ Nickel, Request, Response };
fn stub_3rd_party_function() -> String {
"hello world".to_string()
}
fn main() {
let mut server = Nickel::new();
// assume that the variable **must** be instantiated like this
let hello_text : String = stub_3rd_party_function();
fn hello_handler (_request: &Request, response: &mut Response) -> () {
response.send(hello_text.as_slice());
}
server.get("/hello", hello_handler);
server.listen(Ipv4Addr(0,0,0,0), 6767);
}
Results in the following error:
src/test.rs:12:23: 12:33 error: can't capture dynamic environment in a fn item; use the || { ... } closure form instead
src/test.rs:12 response.send(hello_text);
^~~~~~~~~~
src/test.rs:12:23: 12:33 error: unresolved name `hello_text`.
src/test.rs:12 response.send(hello_text);
^~~~~~~~~~
error: aborting due to 2 previous errors
Now, I switch from a standard function to a closure function instead:
extern crate nickel;
use std::io::net::ip::Ipv4Addr;
use nickel::{ Nickel, Request, Response };
fn stub_3rd_party_function() -> String {
"hello world".to_string()
}
fn main() {
let mut server = Nickel::new();
// assume that the variable **must** be instantiated like this
let hello_text : String = stub_3rd_party_function();
let hello_handler = |_request: &Request, response: &mut Response| -> () {
response.send(hello_text.as_slice());
};
server.get("/hello", hello_handler);
server.listen(Ipv4Addr(0,0,0,0), 6767);
}
Results in a different error:
src/test.rs:21:30: 21:43 error: mismatched types: expected `fn(&nickel::request::Request<'_>, &mut nickel::response::Response<'_,'_>)` but found `|&nickel::request::Request<'_>, &mut nickel::response::Response<'_,'_>|` (expected extern fn but found fn)
src/test.rs:21 server.get("/hello", hello_handler);
^~~~~~~~~~~~~
error: aborting due to previous error
Is there perhaps a way to "wrap" the closed over function with a normal one?
Since the library that I am using expects a standard function instead of a closure,
I cannot use a closure.
But if I do not use a closure, I cannot close over variables that are defined within the outer function, fn main () { ... }... and thus getting stuck here.
Note that above, I am using a string, hello_text, for the purposes of providing a concise code example.
In this case using a static variable would suffice. However, static variables will not fix it for me, as I need to be able to assign a variable from within a fn main() the result of a function call, and then use that within my inner handler function.
It says that because it’s the simple truth: a function cannot capture variables; if you put a function inside another function, that the function is inside the function rather than outside is purely a matter of namespacing and making it absolutely private and inaccessible to anything else. If you want such variable capturing, you must use a closure.
In your specific case, functions are the only way. You should consider your code to be this (I would write it this way, too, to reduce indentation if nothing else):
fn hello_handler(_request: &Request, response: &mut Response) {
response.send(hello_text);
}
fn main() {
let mut server = Nickel::new();
let hello_text = "hello world";
server.get("/hello", hello_handler);
server.listen(Ipv4Addr(0, 0, 0, 0), 6767);
}
As you can see with this way of expressing it, hello_text is evidently inaccessible from hello_handler. There are sound technical reasons why it must be so, also—each request is handled in its own task. In this particular case, a static is the solution:
static HELLO_TEXT: &'static str = "hello world";