Rust: How to specify lifetimes in closure arguments? - debugging

Question

I'm writing a parser generator as a project to learn rust, and I'm running into something I can't figure out with lifetimes and closures. Here's my simplified case (sorry it's as complex as it is, but I need to have the custom iterator in the real version and it seems to make a difference in the compiler's behavior):
Playpen link: http://is.gd/rRm2aa
struct MyIter<'stat, T:Iterator<&'stat str>>{
source: T
}
impl<'stat, T:Iterator<&'stat str>> Iterator<&'stat str> for MyIter<'stat, T>{
fn next(&mut self) -> Option<&'stat str>{
self.source.next()
}
}
struct Scanner<'stat,T:Iterator<&'stat str>>{
input: T
}
impl<'main> Scanner<'main, MyIter<'main,::std::str::Graphemes<'main>>>{
fn scan_literal(&'main mut self) -> Option<String>{
let mut token = String::from_str("");
fn get_chunk<'scan_literal,'main>(result:&'scan_literal mut String,
input: &'main mut MyIter<'main,::std::str::Graphemes<'main>>)
-> Option<&'scan_literal mut String>{
Some(input.take_while(|&chr| chr != "\"")
.fold(result, |&mut acc, chr|{
acc.push_str(chr);
&mut acc
}))
}
get_chunk(&mut token,&mut self.input);
println!("token is {}", token);
Some(token)
}
}
fn main(){
let mut scanner = Scanner{input:MyIter{source:"\"foo\"".graphemes(true)}};
scanner.scan_literal();
}
There are two problems I know of here. First, I have to shadow the 'main lifetime in the get_chunk function (I tried using the one in the impl, but the compiler complains that 'main is undefined inside get_chunk). I think it will still work out because the call to get_chunk later will match the 'main from the impl with the 'main from get_chunk, but I'm not sure that's right.
The second problem is that the &mut acc inside the closure needs to have a lifetime of 'scan_literal in order to work like I want it to (accumulating characters until the first " is encountered for this example). I can't add an explicit lifetime to &mut acc though, and the compiler says its lifetime is limited to the closure itself, and thus I can't return the reference to use in the next iteration of fold. I've gotten the function to compile and run in various other ways, but I don't understand what the problem is here.
My main question is: Is there any way to explicitly specify the lifetime of an argument to a closure? If not, is there a better way to accumulate the string using fold without doing multiple copies?

Answer 1

First, about lifetimes. Functions defined inside other functions are static, they are not connected with their outside code in any way. Consequently, their lifetime parameters are completely independent. You don't want to use 'main as a lifetime parameter for get_chunk() because it will shadow the outer 'main lifetime and give nothing but confusion.
Next, about closures. This expression:
|&mut acc, chr| ...
very likely does not what you really think it does. Closure/function arguments allow irrefutable patterns in them, and & have special meaning in patterns. Namely, it dereferences the value it is matched against, and assigns its identifier to this dereferenced value:
let x: int = 10i;
let p: &int = &x;
match p {
&y => println!("{}", y) // prints 10
}
You can think of & in a pattern as an opposite to & in an expression: in an expression it means "take a reference", in a pattern it means "remove the reference".
mut, however, does not belong to & in patterns; it belongs to the identifier and means that the variable with this identifier is mutable, i.e. you should write not
|&mut acc, chr| ...
but
|& mut acc, chr| ...
You may be interested in this RFC which is exactly about this quirk in the language syntax.
It looks like that you want to do a very strange thing, I'm not sure I understand where you're getting at. It is very likely that you are confusing different string kinds. First of all, you should read the official guide which explains ownership and borrowing and when to use them (you may also want to read the unfinished ownership guide; it will soon get into the main documentation tree), and then you should read strings guide.
Anyway, your problem can be solved in much simpler and generic way:
#[deriving(Clone)]
struct MyIter<'s, T: Iterator<&'s str>> {
source: T
}
impl<'s, T: Iterator<&'s str>> Iterator<&'s str> for MyIter<'s, T>{
fn next(&mut self) -> Option<&'s str>{ // '
self.source.next()
}
}
#[deriving(Clone)]
struct Scanner<'s, T: Iterator<&'s str>> {
input: T
}
impl<'m, T: Iterator<&'m str>> Scanner<'m, T> { // '
fn scan_literal(&mut self) -> Option<String>{
fn get_chunk<'a, T: Iterator<&'a str>>(input: T) -> Option<String> {
Some(
input.take_while(|&chr| chr != "\"")
.fold(String::new(), |mut acc, chr| {
acc.push_str(chr);
acc
})
)
}
let token = get_chunk(self.input.by_ref());
println!("token is {}", token);
token
}
}
fn main(){
let mut scanner = Scanner{
input: MyIter {
source: "\"foo\"".graphemes(true)
}
};
scanner.scan_literal();
}
You don't need to pass external references into the closure; you can generate a String directly in fold() operation. I also generified your code and made it more idiomatic.
Note that now impl for Scanner also works with arbitrary iterators returning &str. It is very likely that you want to write this instead of specializing Scanner to work only with MyIter with Graphemes inside it. by_ref() operation turns &mut I where I is an Iterator<T> into J, where J is an Iterator<T>. It allows further chaining of iterators even if you only have a mutable reference to the original iterator.
By the way, your code is also incomplete; it will only return Some("") because the take_while() will stop at the first quote and won't scan further. You should rewrite it to take initial quote into account.