I am a Rust newbie, and don't understand all the rules for lifetime elision and inference. I can't seem to get returning a reference into an argument from a closure to work, and the errors don't help much for someone with my amount of knowledge.
I can use a proper function with lifetime annotations in place of a closure, but can't figure out a way to annotate these lifetimes on the closure.
struct Person<'a> {
name: &'a str,
}
impl<'a> Person<'a> {
fn map<F, T>(&'a self, closure: F) -> T
where F: Fn(&'a Person) -> T
{
closure(self)
}
}
fn get_name<'a>(person: &'a Person) -> &'a str {
person.name
}
fn main() {
let p = Person { name: "hello" };
let s: &str = p.map(|person| person.name); // Does not work
// let s: &str = p.map(get_name); // Works
println!("{:?}", s);
}
And here is the compiler error:
error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
--> src/main.rs:19:34
|
19 | let s: &str = p.map(|person| person.name); // Does not work
| ^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the body at 19:33...
--> src/main.rs:19:34
|
19 | let s: &str = p.map(|person| person.name); // Does not work
| ^^^^^^^^^^^
note: ...so that expression is assignable (expected &str, found &str)
--> src/main.rs:19:34
|
19 | let s: &str = p.map(|person| person.name); // Does not work
| ^^^^^^^^^^^
note: but, the lifetime must be valid for the method call at 19:18...
--> src/main.rs:19:19
|
19 | let s: &str = p.map(|person| person.name); // Does not work
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: ...so that pointer is not dereferenced outside its lifetime
--> src/main.rs:19:19
|
19 | let s: &str = p.map(|person| person.name); // Does not work
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
What is happening here, and how to change the closure for it to work?
I think I was confusing the lifetime of reference to the Person with the lifetime parameter of the struct (of the &str reference). This compiles:
struct Person<'a> {
name: &'a str,
}
impl<'a> Person<'a> {
fn map<F, T>(&self, closure: F) -> T
where F: Fn(&Person<'a>) -> T
{
closure(self)
}
}
fn get_name<'a>(person: &Person<'a>) -> &'a str {
person.name
}
fn main() {
let p = Person { name: "hello" };
println!("{:?}", p.map(|person| person.name));
println!("{:?}", p.map(get_name));
}
I'm not sure if my understanding of the lifetime annotations is correct, and perhaps someone else can pick apart the compiler error above. The language is extremely confusing.
Related
This question already has answers here:
Return local String as a slice (&str)
(7 answers)
Closed 8 months ago.
I'm trying to return a Result<(), &str> in rust, where the &str has embedded data about any errors which have occurred. For example, say I have the following code:
struct Foo {
pub mynum :i32,
}
impl Foo {
fn do_something(&self) -> Result<(), &str> {
if self.mynum % 12 == 0 {
return Err(&format!("error! mynum is {}", self.mynum))
}
Ok(())
}
}
fn main() {
let foo_instance = Foo{
mynum: 36,
};
let result = foo_instance.do_something().unwrap();
println!("{:?}",result)
}
If I run this in the rust playground, I get
error[E0515]: cannot return value referencing temporary value
--> src/main.rs:9:20
|
9 | return Err(&format!("error! mynum is {}", self.mynum))
| ^^^^^-----------------------------------------^
| | |
| | temporary value created here
| returns a value referencing data owned by the current function
which isn't desired.
How do I tell the Rust compiler that to create a &str with lifetime 'a where 'a is the lifetime of self? I don't want to use 'static if possible, and I don't want to load Foo with extra members..
You should not return a &str in this case because the underlying object the &str is referencing gets dropped when the function terminates. Essentially, you are attempting to return a reference to a temporary value which gets deleted. See this article on differences between String and &str.
String is probably what you want instead. This compiles:
fn do_something(&self) -> Result<(), String> {
if self.mynum % 12 == 0 {
return Err(format!("error! mynum is {}", self.mynum));
}
Ok(())
}
I'm trying to implement a linked list to understand smart pointers in Rust. I defined a Node:
use std::{cell::RefCell, rc::Rc};
struct Node {
val: i32,
next: Option<Rc<RefCell<Node>>>,
}
and iterate like
fn iterate(node: Option<&Rc<RefCell<Node>>>) -> Vec<i32> {
let mut p = node;
let mut result = vec![];
loop {
if p.is_none() {
break;
}
result.push(p.as_ref().unwrap().borrow().val);
p = p.as_ref().unwrap().borrow().next.as_ref();
}
result
}
the compiler reports an error:
error[E0716]: temporary value dropped while borrowed
--> src/main.rs:27:13
|
27 | p = p.as_ref().unwrap().borrow().next.as_ref();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -
| | |
| | temporary value is freed at the end of this statement
| | ... and the borrow might be used here, when that temporary is dropped and runs the destructor for type `std::cell::Ref<'_, Node>`
| creates a temporary which is freed while still in use
| a temporary with access to the borrow is created here ...
|
= note: consider using a `let` binding to create a longer lived value
What happened? Can't we use a reference to iterate on a node defined this way?
Instead of assigning p the borrowed reference, you need to clone the Rc:
use std::cell::RefCell;
use std::rc::Rc;
struct Node {
val: i32,
next: Option<Rc<RefCell<Node>>>,
}
fn iterate(node: Option<Rc<RefCell<Node>>>) -> Vec<i32> {
let mut p = node;
let mut result = vec![];
loop {
let node = match p {
None => break,
Some(ref n) => Rc::clone(n), // Clone the Rc
};
result.push(node.as_ref().borrow().val); //works because val is Copy
p = match node.borrow().next {
None => None,
Some(ref next) => Some(Rc::clone(next)), //clone the Rc
};
}
result
}
fn main() {
let node = Some(Rc::new(RefCell::new(Node {
val: 0,
next: Some(Rc::new(RefCell::new(Node { val: 1, next: None }))),
})));
let result = iterate(node);
print!("{:?}", result)
}
This is necessary because you are trying to use a variable with a shorter lifespan in a context that requires a longer lifespan. The result of p.as_ref().unwrap().borrow() is dropped (i.e. freed, de-allocated) after the loop iteration, but you are trying to use its members in the next loop (this is called use after free and one of the design goals of Rust is to prevent that).
The issue is that borrows do not own the object. If you want to use the next as p in the next loop, then p will have to own the object. This can be achieved with Rc (i.e. 'reference counted') and allows for multiple owners in a single thread.
What if the definition of Node::next is Option<Box<RefCell<Node>>>, how to iterate over this list?
Yes, I'm also very confused with RefCell, without RefCell we can iterate over list using reference only, but will fail with RefCell. I even tried to add a vector of Ref to save the reference, but still can not success.
If you drop the RefCell you can iterate it like this:
struct Node {
val: i32,
next: Option<Box<Node>>,
}
fn iterate(node: Option<Box<Node>>) -> Vec<i32> {
let mut result = vec![];
let mut next = node.as_ref().map(|n| &**n);
while let Some(n) = next.take() {
result.push(n.val);
let x = n.next.as_ref().map(|n| &**n);
next = x;
}
result
}
fn main() {
let node = Some(Box::new(Node {
val: 0,
next: Some(Box::new(Node { val: 1, next: None })),
}));
let result = iterate(node);
print!("{:?}", result)
}
Maybe it's possible with a RefCell as well, but I was not able to work around the lifetime issues.
I bring a little different code from above answer, one match expression in the loop.
fn iterate(node: Option<Rc<RefCell<ListNode>>>) -> Vec<i32>{
let mut result = vec![];
let mut p = match node{
Some(x) => Rc::clone(&x),
None => return result,
};
loop {
result.push(p.as_ref().borrow().val); //works because val is Copy
let node = match &p.borrow().next{
Some(x) => Rc::clone(&x),
None => break,
};
p = node;
}
result
}
I have a closure that captures and modifies its environment. I want to pass this closure to a function that accepts closures:
fn main() {
let mut integer = 5;
let mut closure_variable = || -> i32 {
integer += 1;
integer
};
execute_closure(&mut closure_variable);
}
fn execute_closure(closure_argument: &mut Fn() -> i32) {
let result = closure_argument();
println!("Result of closure: {}", result);
}
Because the closure modifies its environment, this fails:
error[E0525]: expected a closure that implements the `Fn` trait, but this closure only implements `FnMut`
--> src/main.rs:3:32
|
3 | let mut closure_variable = || -> i32 {
| ________________________________^
4 | | integer += 1;
5 | | integer
6 | | };
| |_____^
7 | execute_closure(&mut closure_variable);
| --------------------- the requirement to implement `Fn` derives from here
|
note: closure is `FnMut` because it mutates the variable `integer` here
--> src/main.rs:4:9
|
4 | integer += 1;
| ^^^^^^^
As I understand from When does a closure implement Fn, FnMut and FnOnce?, this means that my closure actually is expanded to a struct that implements the trait FnMut. This trait is mutable, meaning calling the function changes the (implicit) object. I think this correct, because the variable integer should be modified after calling execute_closure().
How do I convince the compiler this is okay and that I actually want to call a FnMut function? Or is there something fundamentally wrong with how I use Rust in this example?
If you can change the function that accepts the closure...
Accept a FnMut instead of a Fn:
fn main() {
let mut integer = 5;
execute_closure(|| {
integer += 1;
integer
});
}
fn execute_closure<F>(mut closure_argument: F)
where
F: FnMut() -> i32,
{
let result = closure_argument();
println!("Result of closure: {}", result);
}
If you can not change the function that accepts the closure...
Use interior mutability provided by types like Cell or RefCell:
use std::cell::Cell;
fn main() {
let integer = Cell::new(5);
execute_closure(|| {
integer.set(integer.get() + 1);
integer.get()
});
}
fn execute_closure<F>(closure_argument: F)
where
F: Fn() -> i32,
{
let result = closure_argument();
println!("Result of closure: {}", result);
}
Or is there something fundamentally wrong with how I use Rust in this example?
Perhaps. An argument of type &mut Fn() -> i32 cannot mutate the variables it has closed over, so the error message makes sense to me.
It's kind of similar to the type &mut &u8 — you could alter the outer reference to point to another immutable reference, but you cannot "ignore" the inner immutability and change the numeric value.
Aside:
The original code uses dynamic dispatch because there is a trait object that provides indirection. In many cases you'd see this version that I posted above, which uses static dispatch and can be monomorphized. I've also inlined the closure as that's the normal syntax.
Here's the original version with just enough changes to work:
fn main() {
let mut integer = 5;
let mut closure_variable = || -> i32 {
integer += 1;
integer
};
execute_closure(&mut closure_variable);
}
fn execute_closure(closure_argument: &mut FnMut() -> i32) {
let result = closure_argument();
println!("Result of closure: {}", result);
}
This code is an inefficient way of producing a unique set of items from an iterator. To accomplish this, I am attempting to use a Vec to keep track of values I've seen. I believe that this Vec needs to be owned by the innermost closure:
fn main() {
let mut seen = vec![];
let items = vec![vec![1i32, 2], vec![3], vec![1]];
let a: Vec<_> = items
.iter()
.flat_map(move |inner_numbers| {
inner_numbers.iter().filter_map(move |&number| {
if !seen.contains(&number) {
seen.push(number);
Some(number)
} else {
None
}
})
})
.collect();
println!("{:?}", a);
}
However, compilation fails with:
error[E0507]: cannot move out of captured outer variable in an `FnMut` closure
--> src/main.rs:8:45
|
2 | let mut seen = vec![];
| -------- captured outer variable
...
8 | inner_numbers.iter().filter_map(move |&number| {
| ^^^^^^^^^^^^^^ cannot move out of captured outer variable in an `FnMut` closure
This is a little surprising, but isn't a bug.
flat_map takes a FnMut as it needs to call the closure multiple times. The code with move on the inner closure fails because that closure is created multiple times, once for each inner_numbers. If I write the closures in explicit form (i.e. a struct that stores the captures and an implementation of one of the closure traits) your code looks (a bit) like
struct OuterClosure {
seen: Vec<i32>
}
struct InnerClosure {
seen: Vec<i32>
}
impl FnMut(&Vec<i32>) -> iter::FilterMap<..., InnerClosure> for OuterClosure {
fn call_mut(&mut self, (inner_numbers,): &Vec<i32>) -> iter::FilterMap<..., InnerClosure> {
let inner = InnerClosure {
seen: self.seen // uh oh! a move out of a &mut pointer
};
inner_numbers.iter().filter_map(inner)
}
}
impl FnMut(&i32) -> Option<i32> for InnerClosure { ... }
Which makes the illegality clearer: attempting to move out of the &mut OuterClosure variable.
Theoretically, just capturing a mutable reference is sufficient, since the seen is only being modified (not moved) inside the closure. However things are too lazy for this to work...
error: lifetime of `seen` is too short to guarantee its contents can be safely reborrowed
--> src/main.rs:9:45
|
9 | inner_numbers.iter().filter_map(|&number| {
| ^^^^^^^^^
|
note: `seen` would have to be valid for the method call at 7:20...
--> src/main.rs:7:21
|
7 | let a: Vec<_> = items.iter()
| _____________________^
8 | | .flat_map(|inner_numbers| {
9 | | inner_numbers.iter().filter_map(|&number| {
10| | if !seen.contains(&number) {
... |
17| | })
18| | .collect();
| |__________________^
note: ...but `seen` is only valid for the lifetime as defined on the body at 8:34
--> src/main.rs:8:35
|
8 | .flat_map(|inner_numbers| {
| ___________________________________^
9 | | inner_numbers.iter().filter_map(|&number| {
10| | if !seen.contains(&number) {
11| | seen.push(number);
... |
16| | })
17| | })
| |_________^
Removing the moves makes the closure captures work like
struct OuterClosure<'a> {
seen: &'a mut Vec<i32>
}
struct InnerClosure<'a> {
seen: &'a mut Vec<i32>
}
impl<'a> FnMut(&Vec<i32>) -> iter::FilterMap<..., InnerClosure<??>> for OuterClosure<'a> {
fn call_mut<'b>(&'b mut self, inner_numbers: &Vec<i32>) -> iter::FilterMap<..., InnerClosure<??>> {
let inner = InnerClosure {
seen: &mut *self.seen // can't move out, so must be a reborrow
};
inner_numbers.iter().filter_map(inner)
}
}
impl<'a> FnMut(&i32) -> Option<i32> for InnerClosure<'a> { ... }
(I've named the &mut self lifetime in this one, for pedagogical purposes.)
This case is definitely more subtle. The FilterMap iterator stores the closure internally, meaning any references in the closure value (that is, any references it captures) have to be valid as long as the FilterMap values are being thrown around, and, for &mut references, any references have to be careful to be non-aliased.
The compiler can't be sure flat_map won't, e.g. store all the returned iterators in a Vec<FilterMap<...>> which would result in a pile of aliased &muts... very bad! I think this specific use of flat_map happens to be safe, but I'm not sure it is in general, and there's certainly functions with the same style of signature as flat_map (e.g. map) would definitely be unsafe. (In fact, replacing flat_map with map in the code gives the Vec situation I just described.)
For the error message: self is effectively (ignoring the struct wrapper) &'b mut (&'a mut Vec<i32>) where 'b is the lifetime of &mut self reference and 'a is the lifetime of the reference in the struct. Moving the inner &mut out is illegal: can't move an affine type like &mut out of a reference (it would work with &Vec<i32>, though), so the only choice is to reborrow. A reborrow is going through the outer reference and so cannot outlive it, that is, the &mut *self.seen reborrow is a &'b mut Vec<i32>, not a &'a mut Vec<i32>.
This makes the inner closure have type InnerClosure<'b>, and hence the call_mut method is trying to return a FilterMap<..., InnerClosure<'b>>. Unfortunately, the FnMut trait defines call_mut as just
pub trait FnMut<Args>: FnOnce<Args> {
extern "rust-call" fn call_mut(&mut self, args: Args) -> Self::Output;
}
In particular, there's no connection between the lifetime of the self reference itself and the returned value, and so it is illegal to try to return InnerClosure<'b> which has that link. This is why the compiler is complaining that the lifetime is too short to be able to reborrow.
This is extremely similar to the Iterator::next method, and the code here is failing for basically the same reason that one cannot have an iterator over references into memory that the iterator itself owns. (I imagine a "streaming iterator" (iterators with a link between &mut self and the return value in next) library would be able to provide a flat_map that works with the code nearly written: would need "closure" traits with a similar link.)
Work-arounds include:
the RefCell suggested by Renato Zannon, which allows seen to be borrowed as a shared &. The desugared closure code is basically the same other than changing the &mut Vec<i32> to &Vec<i32>. This change means "reborrow" of the &'b mut &'a RefCell<Vec<i32>> can just be a copy of the &'a ... out of the &mut. It's a literal copy, so the lifetime is retained.
avoiding the laziness of iterators, to avoid returning the inner closure, specifically.collect::<Vec<_>>()ing inside the loop to run through the whole filter_map before returning.
fn main() {
let mut seen = vec![];
let items = vec![vec![1i32, 2], vec![3], vec![1]];
let a: Vec<_> = items
.iter()
.flat_map(|inner_numbers| {
inner_numbers
.iter()
.filter_map(|&number| if !seen.contains(&number) {
seen.push(number);
Some(number)
} else {
None
})
.collect::<Vec<_>>()
.into_iter()
})
.collect();
println!("{:?}", a);
}
I imagine the RefCell version is more efficient.
It seems that the borrow checker is getting confused at the nested closures + mutable borrow. It might be worth filing an issue. Edit: See huon's answer for why this isn't a bug.
As a workaround, it's possible to resort to RefCell here:
use std::cell::RefCell;
fn main() {
let seen = vec![];
let items = vec![vec![1i32, 2], vec![3], vec![1]];
let seen_cell = RefCell::new(seen);
let a: Vec<_> = items
.iter()
.flat_map(|inner_numbers| {
inner_numbers.iter().filter_map(|&number| {
let mut borrowed = seen_cell.borrow_mut();
if !borrowed.contains(&number) {
borrowed.push(number);
Some(number)
} else {
None
}
})
})
.collect();
println!("{:?}", a);
}
I came across this question after I ran into a similar issue, using flat_map and filter_map. I solved it by moving the filter_map outside the flat_map closure.
Using your example:
let a: Vec<_> = items
.iter()
.flat_map(|inner_numbers| inner_numbers.iter())
.filter_map(move |&number| {
if !seen.contains(&number) {
seen.push(number);
Some(number)
} else {
None
}
})
.collect();
Before Rust 1.0, I could write a structure using this obsolete closure syntax:
struct Foo {
pub foo: |usize| -> usize,
}
Now I can do something like:
struct Foo<F: FnMut(usize) -> usize> {
pub foo: F,
}
But then what's the type of a Foo object I create?
let foo: Foo<???> = Foo { foo: |x| x + 1 };
I could also use a reference:
struct Foo<'a> {
pub foo: &'a mut FnMut(usize) -> usize,
}
I think this is slower because
the pointer dereference
there's no specialization for the type of FnMut that actually ends up being used
Complementing the existing answer with some more code for demonstration purposes:
Unboxed closure
Use a generic type:
struct Foo<F>
where
F: Fn(usize) -> usize,
{
pub foo: F,
}
impl<F> Foo<F>
where
F: Fn(usize) -> usize,
{
fn new(foo: F) -> Self {
Self { foo }
}
}
fn main() {
let foo = Foo { foo: |a| a + 1 };
(foo.foo)(42);
(Foo::new(|a| a + 1).foo)(42);
}
Boxed trait object
struct Foo {
pub foo: Box<dyn Fn(usize) -> usize>,
}
impl Foo {
fn new(foo: impl Fn(usize) -> usize + 'static) -> Self {
Self { foo: Box::new(foo) }
}
}
fn main() {
let foo = Foo {
foo: Box::new(|a| a + 1),
};
(foo.foo)(42);
(Foo::new(|a| a + 1).foo)(42);
}
Trait object reference
struct Foo<'a> {
pub foo: &'a dyn Fn(usize) -> usize,
}
impl<'a> Foo<'a> {
fn new(foo: &'a dyn Fn(usize) -> usize) -> Self {
Self { foo }
}
}
fn main() {
let foo = Foo { foo: &|a| a + 1 };
(foo.foo)(42);
(Foo::new(&|a| a + 1).foo)(42);
}
Function pointer
struct Foo {
pub foo: fn(usize) -> usize,
}
impl Foo {
fn new(foo: fn(usize) -> usize) -> Self {
Self { foo }
}
}
fn main() {
let foo = Foo { foo: |a| a + 1 };
(foo.foo)(42);
(Foo::new(|a| a + 1).foo)(42);
}
what's the type of a Foo object I create?
It's an unnameable, automatically generated type.
I could also use a reference [...] slower because [...] the pointer deref [...] no specialization
Perhaps, but it can be much easier on the caller.
See also:
How do I call a function through a member variable?
Returning a closure from a function
How to return an anonymous type from a trait method without using Box?
Closures as a type in a Rust struct
Types of unboxed closures being unique to each
Why does passing a closure to function which accepts a function pointer not work?
What does "dyn" mean in a type?
For what type you'd use in your third code snippet, there isn't one; closure types are anonymous and cannot be directly named. Instead, you'd write:
let foo = Foo { foo: |x| x + 1 };
If you're writing code in a context where you need to specify that you want a Foo, you'd write:
let foo: Foo<_> = Foo { foo: |x| x + 1 };
The _ tells the type system to infer the actual generic type for you.
The general rule of thumb as to which to use, in descending order:
Generic parameters: struct Foo<F: FnMut(usize) -> usize>. This is the most efficient, but it does mean that a specific Foo instance can only ever store one closure, since every closure has a different concrete type.
Trait references: &'a mut dyn FnMut(usize) -> usize. There's a pointer indirection, but now you can store a reference to any closure that has a compatible call signature.
Boxed closures: Box<dyn FnMut(usize) -> usize>. This involves allocating the closure on the heap, but you don't have to worry about lifetimes. As with a reference, you can store any closure with a compatible signature.
Before Rust 1.0
Closures that used the || syntax were references to closures stored on the stack, making them equivalent to &'a mut FnMut(usize) -> usize. Old-style procs were heap-allocated and were equivalent to Box<dyn FnOnce(usize) -> usize> (you can only call a proc once).