"Recycling" items in iterators for better performance - performance

Question

I have a file that contains multiple instances of some complex data type (think of a trajectory of events). The API to read this file is written in C and I don't have much control over it. To expose it to Rust, I implemented the following interface:
// a single event read from the file
struct Event {
a: u32,
b: f32,
}
// A handle to the file used for I/O
struct EventFile;
impl EventFile {
fn open() -> Result<EventFile, Error> {
unimplemented!()
}
// read the next step of the trajectory into event
fn read(&self, event: &mut Event) -> Result<(), Error> {
event.a = unimplemented!();
event.b = unimplemented!();
}
}
To access the file contents, I could call the read function until it returns an Err similar to this:
let event_file = EventFile::open();
let mut event = Event::new();
let mut result = event_file.read(&mut event);
while let Ok(_) = result {
println!("{:?}", event);
result = event_file.read(&mut event);
}
Because event is reused for each call of read, there's no repeated allocation/deallocation of memory which hopefully results in some performance improvement (the event struct is much bigger in the actual implementation).
Now, It would be nice to be able to access this data through an iterator. However, to my understanding, this means that I have to create a new instance of Event each time the iterator yields - because I cannot reuse the event inside with an iterator. And this will hurt the performance:
struct EventIterator {
event_file: EventFile,
}
impl Iterator for EventIterator {
type Item = Event;
fn next(&mut self) -> Option<Event> {
let mut event = Event::new(); // costly allocation
let result = self.event_file.read(&mut event);
match result {
Ok(_) => Some(event),
Err(_) => None,
}
}
}
let it = EventIterator { event_file };
it.map(|event| unimplemented!())
Is there a way to somehow "recycle" or "reuse" events inside the iterator? Or is this a concept that is simply not transferable to Rust and I have to live with worse performance using iterators in this case?

Answer 1

You can "recycle" items between iterations by wrapping the Item in a reference counter. The idea here is that if the caller keeps the item around between iterations, the iterator allocates a new object and returns that new object. If the item is dropped by the caller before the next iteration begins, the item is recycled. This is ensured by std::rc::Rc::get_mut(), which will only return a reference if the reference-count is exactly 1.
This has the downside that your Iterator yields Rc<Foo> instead of Foo. There is also the added code-complexity and (maybe) some runtime-cost due to the reference-counting (which may get elided completely if the compiler can prove that).
You will, therefore, need to measure if this actually gets you a performance win. Allocating a new object on every single iteration may seem costly, but allocators are good at this...
Something to the tune of
use std::rc::Rc;
#[derive(Default)]
struct FoobarIterator {
item: Rc<String>,
}
impl Iterator for FoobarIterator {
type Item = Rc<String>;
fn next(&mut self) -> Option<Self::Item> {
let item = match Rc::get_mut(&mut self.item) {
Some(item) => {
// This path is only taken if the caller
// did not keep the item around
// so we are the only reference-holder!
println!("Item is re-used!");
item
},
None => {
// Let go of the item (the caller gets to keep it)
// and create a new one
println!("Creating new item!");
self.item = Rc::new(String::new());
Rc::get_mut(&mut self.item).unwrap()
}
};
// Create the item, possible reusing the same allocation...
item.clear();
item.push('a');
Some(Rc::clone(&self.item))
}
}
fn main() {
// This will only print "Item is re-used"
// because `item` is dropped before the next cycle begins
for item in FoobarIterator::default().take(5) {
println!("{}", item);
}
// This will allocate new objects every time
// because the Vec retains ownership.
let _: Vec<_> = FoobarIterator::default().take(5).collect();
}

Answer 2

The compiler (or LLVM) will most likely employ return value optimization in this case, so you do not need to prematurely optimize by yourself.
See this Godbolt example, particularly lines 43 to 47. My comprehension of Assembly is limited, but it seems that next() simply writes the Event value to the memory passed by the caller via a pointer (initially in rdi). In subsequent loop iterations this memory place can be reused.
Note that you get a much longer assembly output (which I did not analyze in depth) if you compile without the -O flag (e.g. when building in the "debug" mode as opposed to "release").