Expected concrete lifetime, found bound lifetime parameter when storing a closure [duplicate] - closures

I'm encountering a strange pair of errors while trying to compile my Rust code below. In searching for others with similar problems, I came across another question with the same combination of (seemingly opposing) errors, but couldn't generalize the solution from there to my problem.
Basically, I seem to be missing a subtlety in Rust's ownership system. In trying to compile the (very pared down) code here:
struct Point {
x: f32,
y: f32,
}
fn fold<S, T, F>(item: &[S], accum: T, f: F) -> T
where
F: Fn(T, &S) -> T,
{
f(accum, &item[0])
}
fn test<'a>(points: &'a [Point]) -> (&'a Point, f32) {
let md = |(q, max_d): (&Point, f32), p: &'a Point| -> (&Point, f32) {
let d = p.x + p.y; // Standing in for a function call
if d > max_d {
(p, d)
} else {
(q, max_d)
}
};
fold(&points, (&Point { x: 0., y: 0. }, 0.), md)
}
I get the following error messages:
error[E0631]: type mismatch in closure arguments
--> src/main.rs:23:5
|
14 | let md = |(q, max_d): (&Point, f32), p: &'a Point| -> (&Point, f32) {
| ---------------------------------------------------------- found signature of `for<'r> fn((&'r Point, f32), &'a Point) -> _`
...
23 | fold(&points, (&Point { x: 0., y: 0. }, 0.), md)
| ^^^^ expected signature of `for<'r> fn((&Point, f32), &'r Point) -> _`
|
= note: required by `fold`
error[E0271]: type mismatch resolving `for<'r> <[closure#src/main.rs:14:14: 21:6] as std::ops::FnOnce<((&Point, f32), &'r Point)>>::Output == (&Point, f32)`
--> src/main.rs:23:5
|
23 | fold(&points, (&Point { x: 0., y: 0. }, 0.), md)
| ^^^^ expected bound lifetime parameter, found concrete lifetime
|
= note: required by `fold`
(A Rust Playground link for this code, for convenience.)
It seems to me that the function I'm supplying to fold should type-check properly... what am I missing here and how can I go about fixing it?

The short version is that there's a difference between the lifetimes that are inferred if the closure is written inline or stored as a variable. Write the closure inline and remove all the extraneous types:
fn test(points: &[Point]) -> (&Point, f32) {
let init = points.first().expect("No initial");
fold(&points, (init, 0.), |(q, max_d), p| {
let d = 12.;
if d > max_d {
(p, d)
} else {
(q, max_d)
}
})
}
If you truly must have the closure out-of-band, review How to declare a lifetime for a closure argument?.
Additionally, I had to pull the first value from the input array — you can't return a reference to a local variable. There's no need for lifetime parameters on the method; they will be inferred.
To actually get the code to compile, you need to provide more information about the fold method. Specifically, you have to indicate that the reference passed to the closure has the same lifetime as the argument passed in. Otherwise, it could just be a reference to a local variable:
fn fold<'a, S, T, F>(item: &'a [S], accum: T, f: F) -> T
where
F: Fn(T, &'a S) -> T,
{
f(accum, &item[0])
}
The related Rust issue is #41078.

Related

How to constrain the return type of a function with an arbitrary amount of arguments?

I would like to create a record type with a type annotation, with the constraint that the annotation must be a function that returns a certain type. Consider this example below:
type Foo<'Function> = {
Function: 'Function
}
I would 'Function to be able to be any function, as long as it returns for example int, so that
{Function = (fun a b c d e f g -> 2)}
{Function = (fun a -> 2)}
work, but
{Function = (fun a -> "")}
would be invalid.
As far as i understand F# functions, they are a nested structure of tuples, where the most inner second tuple field is the return type of the function, which is the reason why neither of these:
type Foo<('Input -> int)> = {
Function: ('Input -> int)
}
type ReturnsInt = FSharpFunc<'Input,int>
will work for any function with more than one argument, because the result type is encapsulated in the second annotation in either the signature or FSharpFunc<_,_>. Is there a way to realize this as a type annotation?
Edit
As Fyodor suggested in the comments below, this can be overcome by using only functions that have tupled arguments, leading to the function annotation being
FSharpFunc<argumentTuples,ReturnType>, e.g.:
type Foo<'ArgumentTuples> = {
Function: 'ArgumentTuples -> int
}
{Function = (fun (a,b,c,d,e,f,g) -> 2)} // works
{Function = (fun a -> 2)} // works
{Function = (fun a -> "")} // wont work
While this approach has its own problems and is not the specific answer to the original question, this workaround/workflow adaptation might be enough for me. I'll leave this question up (can't mark the comment as answer anyways)
Perhaps I'm missing something here, but why not simply define your type as:
type Foo<'InputType> = {
Function: ('InputType -> int)
}
Or more generic:
type Foo<'InputType, 'OutputType> = {
Function: ('InputType -> 'OutputType)
}
Edit: Ok, I see the problem.
So to use Fyodors solution, do something like this:
type Foo<'InputType> = {
Function: ('InputType -> int)
}
let add a b = a + b
let myFunction = {
Function = (fun (a,b) -> add a b)
}
Something along these lines might work for you, although you should be careful about not loosing the general picture. The branching may explode if there is a lot of complexity in your data model.
module FunctionTests
//define function types templates
type F1<'a> = 'a -> int
type F2<'a, 'b> = 'a -> 'b -> int
type F3<'a, 'b, 'c> = 'a -> 'b -> 'c -> int
type F4<'a, 'b, 'c, 'd> = 'a -> 'b -> 'c -> 'd -> int
//define actual functions
let f1 : F1<string> = fun x ->
printf "calling f1: %s\n" x
1
let f2 : F2<string, string> = fun x y ->
printf "calling f2: %s; %s\n" x y
2
let f3 : F3<string, string, int> = fun x y z ->
printf "calling f2: %s; %s; %d\n" x y z
3
let f4 : F4<string, string, int, int> = fun x y z v ->
printf "calling f2: %s; %s; %d; %d\n" x y z v
4
//define DU limiting to a subset of functions
type FunctionChooser =
| FF1 of F1<string>
| FF2 of F2<string, string>
| FF3 of F3<string, string, int>
| FF4 of F4<string, string, int, int>
//define a record with the DU
type FunctionRec = {Function : FunctionChooser}
//testing harness
let callFunction (functiondefs : FunctionRec) data otherdata intparam1 intparam2 =
match functiondefs.Function with
| FF1 fn -> fn data
| FF2 fn -> fn data otherdata
| FF3 fn -> fn data otherdata intparam1
| FF4 fn -> fn data otherdata intparam1 intparam2
//tests
let res1 = callFunction {Function=FF1 f1} "somedata" "otherdata" 13 14
let res2 = callFunction {Function=FF2 f2} "somedata" "otherdata" 13 14
let res3 = callFunction {Function=FF3 f3} "somedata" "otherdata" 13 14
let res4 = callFunction {Function=FF4 f4} "somedata" "otherdata" 13 14

Improve my nested matches in F# [duplicate]

This question already has answers here:
Getting rid of the "pyramid of doom" in F#
(2 answers)
Closed 5 years ago.
Suppose I want to do repeated map lookups.
In C#, I can use return for a "flat" control-flow:
Thing v = null;
if (a.TryGetValue(key, out v))
{
return v;
}
if (b.TryGetValue(key, out v))
{
return v;
}
if (c.TryGetValue(key, out v))
{
return v;
}
return defaultValue;
It's a bit ugly, but quite readable.
In F#, which I am less familiar with, I would use match expressions:
match a.TryGetValue(key) with
| (true, v) -> v
| _ ->
match b.TryGetValue(key) with
| (true, v) -> v
| _ ->
match c.TryGetValue(key) with
| (true, v) -> v
| _ -> defaultValue
This feels wrong - the code gets more and more nested with each map.
Does F# provide a way to "flatten" this code?
You could slightly change the semantics and run all TryGetValue calls up-front. Then you need just one flat pattern match, because you can pattern match on all the results at the same time and use the or pattern (written using |) to select the first one that succeeded:
match a.TryGetValue(key), b.TryGetValue(key), c.TryGetValue(key) with
| (true, v), _, _
| _, (true, v), _
| _, _, (true, v) -> v
| _ -> defaultValue
This flattens the pattern matching, but you might be doing unnecessary lookups (which probably is not such a big deal, but it's worth noting that this is a change of semantics).
Another option is to use an active pattern - you can define a parameterized active pattern that pattern matches on a dictionary, takes the key as an input parameter and performs the lookup:
let (|Lookup|_|) key (d:System.Collections.Generic.IDictionary<_, _>) =
match d.TryGetValue(key) with
| true, v -> Some v
| _ -> None
Now you can write pattern Lookup <key> <pat> which matches a dictionary when it contains a value matching pattern <pat> with the key <key>. Using this, you can rewrite your pattern matching as:
match a, b, c with
| Lookup key v, _, _
| _, Lookup key v, _
| _, _, Lookup key v -> v
| _ -> defaultValue
The way F# compiler handles this is that it will run the patterns one after another and match the first one that succeeds - so if the first one succeeds, only one lookup gets performed.
When control flow becomes a pain, it is sometimes helpful to transform the problem. Say you have this:
let a = [ (1, "a"); (2, "b") ] |> dict
let b = [ (42, "foo"); (7, "bar") ] |> dict
let key = 8
let defaultValue = "defaultValue"
Then the following shows the intent behind the computation: keep trying to get a value, if all fail, use the default.
[ a; b ]
|> Seq.tryPick (fun d -> let (s, v) = d.TryGetValue key in if s then Some v else None)
|> defaultArg <| defaultValue
The more dictionaries you have, the bigger the benefit.
You can also declare a TryGet function that applies a given function f to key only if the variable ok is false. If ok has become true we do nothing, we just return the given couple as input.
let ATryGetValue key =
if key>20 then (true,2) else (false,-2);
let BTryGetValue key =
if key>10 then (true,1) else (false,-1);
let CTryGetValue key =
if key>0 then (true,0) else (false,0);
let tryGet f (ok, key) =
if not ok then
match f key with
| (true, v) -> (true, v)
| _ -> (false, key)
else
(ok,key)
let res key =
tryGet CTryGetValue (tryGet BTryGetValue (tryGet ATryGetValue (false, key)))
printfn "%A" (res 40)

Different Expressions being generated for "roughly" the same code quotation

given the following type
type Foo = { foo: string; bar: int };;
and the following code quotation
<#fun v x -> { x with foo = v; bar = 99 } #>;;
this will result in
val it : Quotations.Expr<(string -> Foo -> Foo)> =
Lambda (v, Lambda (x, NewRecord (Foo, v, Value (99))))
Which is expected. Also the following code quotation
<#fun v x -> { x with bar = v;foo = "foo" } #>;;
yields the expected result.
val it : Quotations.Expr<(int -> Foo -> Foo)> =
Lambda (v, Lambda (x, NewRecord (Foo, Value ("foo"), v)))
However this (changing the order and assigning the value to the second field)
<#fun v x -> { x with bar = 66;foo = v } #>;;
yields
val it : Quotations.Expr<(string -> Foo -> Foo)> =
Lambda (v, Lambda (x, Let (bar, Value (66), NewRecord (Foo, v, bar))))
a let. But there is no let in the code. Why is this?
Quotations only guarantee that they'll generate expressions with the correct behaviour, not any specific shape.
For example the quotation <## 1 = 2 || 2 = 3 ##> will generate an expression comprising of an if statement (i.e. if 1 = 2 then true else 2 = 3).
Normalising the resulting expressions is a pretty deep rabbit hole, but you can see some basic normalisers here: https://github.com/mavnn/Algebra.Boolean/blob/master/Algebra.Boolean/Transforms.fs
Specifically, check unbind at the end of the file.
let unbind quote =
let rec findLet q =
match q with
| Let (var, value, body) ->
findLet (replaceVar var.Name value body)
| ShapeLambda (v, e) ->
Expr.Lambda(v, findLet e)
| ShapeVar v ->
Expr.Var v
| ShapeCombination (o, es) ->
RebuildShapeCombination(o, es |> List.map findLet)
and replaceVar name value q =
match q with
| Let (v, e, e') ->
if v.Name = name then
findLet (Expr.Let(v, e, e'))
else
Expr.Let(v, replaceVar name value e, replaceVar name value e')
| ShapeLambda (v, e) ->
Expr.Lambda(v, replaceVar name value e)
| ShapeVar v ->
if v.Name = name then
value
else
Expr.Var v
| ShapeCombination (o, es) ->
RebuildShapeCombination(o, es |> List.map (replaceVar name value))
findLet quote
As to why these specific expressions are different? No idea, I'm afraid!
I believe what you are seeing here is a particular case of de-sugaring of the with syntax on records. I think what is happening here it is using the v to capture the value to ensure that the expressions are evaluated in the correct order of the fields. So in this case the let binding is introduce as the passed in parameter is the 2nd value being utilised.
This is from the F# language spec.
Primitive record constructions are an elaborated form in which the
fields appear in the same order as in the record type definition.
Record expressions themselves elaborate to a form that may introduce
local value definitions to ensure that expressions are evaluated in
the same order that the field definitions appear in the original
expression

How can I write a builder that stores a Path?

The Path argument could be immediately converted into a PathBuf, but that seems inefficient. There has to be some way of keeping just a Path, right?
use std::{fs::File, path::Path};
struct Foo {
a: Option<File>,
b: Option<File>,
}
struct FooBuilder<'a> {
a: Option<&'a Path>,
b: Option<&'a Path>,
}
impl<'a> FooBuilder<'a> {
fn new() -> FooBuilder<'a> {
FooBuilder { a: None, b: None }
}
fn a<P: AsRef<Path> + 'a>(&'a mut self, a: P) -> &mut FooBuilder<'a> {
self.a = Some(a.as_ref());
self
}
fn b<P: AsRef<Path> + 'a>(&'a mut self, b: P) -> &mut FooBuilder<'a> {
self.b = Some(b.as_ref());
self
}
fn done(&self) -> Foo {
Foo {
a: match self.a {
Some(path) => Some(File::open(path).unwrap()),
None => None,
},
b: match self.b {
Some(path) => Some(File::open(path).unwrap()),
None => None,
},
}
}
}
fn main() {
let path1 = Path::new("1");
let path2 = Path::new("2");
let _foo = FooBuilder::new().a(path1).b(path2).done();
}
error[E0597]: `a` does not live long enough
--> src/main.rs:19:23
|
13 | impl<'a> FooBuilder<'a> {
| -- lifetime `'a` defined here
...
19 | self.a = Some(a.as_ref());
| --------------^----------
| | |
| | borrowed value does not live long enough
| assignment requires that `a` is borrowed for `'a`
20 | self
21 | }
| - `a` dropped here while still borrowed
error[E0597]: `b` does not live long enough
--> src/main.rs:24:23
|
13 | impl<'a> FooBuilder<'a> {
| -- lifetime `'a` defined here
...
24 | self.b = Some(b.as_ref());
| --------------^----------
| | |
| | borrowed value does not live long enough
| assignment requires that `b` is borrowed for `'a`
25 | self
26 | }
| - `b` dropped here while still borrowed
This works:
use std::{fs::File, path::Path};
struct Foo {
a: Option<File>,
}
struct FooBuilder<'a> {
a: Option<&'a Path>,
}
impl<'a> FooBuilder<'a> {
fn new() -> FooBuilder<'a> {
FooBuilder { a: None }
}
fn a<P>(&mut self, a: &'a P) -> &mut FooBuilder<'a>
where
P: AsRef<Path> + ?Sized,
{
self.a = Some(a.as_ref());
self
}
fn build(&self) -> Foo {
Foo {
a: self.a.map(|path| File::open(path).unwrap()),
}
}
}
fn main() {
let path1 = Path::new("1");
let _foo = FooBuilder::new().a(path1).build();
}
Let's focus on the a method:
fn a<P>(&mut self, a: &'a P) -> &mut FooBuilder<'a>
where
P: AsRef<Path> + ?Sized,
This method accepts a reference to a type that implements AsRef<Path>. That means that we can get a reference to a Path with the same lifetime as the parameter. The other change is to make the Sized bound optional for the type via ?. This means that we can have a reference to something that we don't know how big it is. This is fine as we will know how big the reference itself is.
Let's compare this to your original version:
fn a<P: AsRef<Path> + 'a>(&'a mut self, a: P) -> &mut FooBuilder<'a> {
self.a = Some(a.as_ref());
self
}
Here, the a parameter is passed-by-value into the method a. When you call as_ref, you are implicitly calling it on a reference to the item that is on the stack frame of the method call. The referred-to item will be dropped at the end of the method call, which means that the reference would become invalid. That is the reasoning behind the error: `a` does not live long enough error you were getting.
I also used Option::map to clean up the build method. I renamed it to build because builders should generally have a build method, unless there's a more obvious verb to use (like open).
See also:
What does the question mark mean in a type parameter bound?

How to use closures in a nested map?

I am trying to make a 2-dimensional matrix from a functor that creates each element, and store it as a flat Vec (each row concatenated).
I used nested map (actually a flat_map and a nested map) to create each row and concatenate it. Here is what I tried:
fn make<T, F>(n: usize, m: usize, f: F) -> Vec<T>
where
F: Fn(usize, usize) -> T,
{
(0..m).flat_map(|y| (0..n).map(|x| f(x, y))).collect()
}
fn main() {
let v = make(5, 5, |x, y| x + y);
println!("{:?}", v);
}
Unfortunately, I get an error during compilation:
error[E0597]: `y` does not live long enough
--> src/main.rs:5:45
|
5 | (0..m).flat_map(|y| (0..n).map(|x| f(x, y))).collect()
| --- ^ - - borrowed value needs to live until here
| | | |
| | | borrowed value only lives until here
| | borrowed value does not live long enough
| capture occurs here
How does one use closures in nested maps? I worked around this issue by using a single map on 0..n*m, but I'm still interested in the answer.
In your case the inner closure |x| f(x,y) is a borrowing closure, which takes its environment (y and f) by reference.
The way .flat_map(..) works, it forbids you to keep a reference to y, which is not from the outer scope. Thus we need to have your closure take its environment by value, which is not a problem for y being a usize which is Copy:
(0..m).flat_map(|y| (0..n).map(move |x| f(x, y))).collect()
However, now another problem arises:
error[E0507]: cannot move out of captured outer variable in an `FnMut` closure
--> src/main.rs:5:36
|
1 | fn make<T, F>(n: usize, m: usize, f: F) -> Vec<T>
| - captured outer variable
...
5 | (0..m).flat_map(|y| (0..n).map(move |x| f(x,y))).collect()
| ^^^^^^^^ cannot move out of captured outer variable in an `FnMut` closure
Here, we are trying to move f as well into the closure, which is definitely not possible (unless m is 1, but the compiler cannot know that).
Since f is a Fn(usize, usize) -> T, we could just as well explicitly pass a & reference to it, and & references are Copy:
fn make<T, F>(n: usize, m: usize, f: F) -> Vec<T>
where
F: Fn(usize, usize) -> T,
{
let f_ref = &f;
(0..m)
.flat_map(|y| (0..n).map(move |x| f_ref(x, y)))
.collect()
}
In this case, the closure takes its environment by value, and this environment is composed of y and f_ref, both of them being Copy, everything is well.
Adding to Levans's excellent answer, another way of defining the function would be
fn make<T, F>(n: usize, m: usize, f: F) -> Vec<T>
where
F: Fn(usize, usize) -> T + Copy,
{
(0..m).flat_map(|y| (0..n).map(move |x| f(x, y))).collect()
}
Since we know that |x, y| x + y is a Copy type, f would get copied for every callback that flat_map invokes. I would still prefer Levans's way as this would not be as efficient as copying a reference.

Resources