Skip to content

Conversation

@forki
Copy link
Contributor

@forki forki commented Jun 21, 2017

I really want to discuss #1525 again. We closed it because of RTM preparation.
I know there are related efforts to improve the underlying seq implementation. I think both can work well together.

This PR enables the compiler to rewrite

[1; 2] |> Seq.map (fun x -> x + 2) |> Seq.map (fun x -> x * 2)
["hello"; "world"; "!"] |> Seq.map (fun (y:string) -> y.Length) |> Seq.map (fun x -> x * 3)

into:

Seq.map (fun x -> (x + 2) * 2) [1; 2]
Seq.map (fun x -> x.Length * 3) ["hello"; "world"; "!"]

Future rules:

Seq.filter f (Seq.filter g xs)  ==>  Seq.filter (fun x -> g x && f x)) xs
Seq.iter f (Seq.map g xs)  ==>  Seq.iter (fun x -> f(g x)) xs
Seq.iter f (Seq.filter g xs)  ==>  Seq.iter (fun x -> if g x then f x)) xs

Important point is that we always rewrite things to "smaller" terms so that we know it will terminate.

@isaacabraham
Copy link
Contributor

This is good. I personally prefer to always have chain separate filter expressions together rather than as one big block - this would mean that the compiler no longer punishes me for doing so :-)

@forki
Copy link
Contributor Author

forki commented Jun 24, 2017

Another important thing here is that fusion would also work on fable projects

@KevinRansom
Copy link
Contributor

@forki is this going to remain wip, or should it be closed?

@forki
Copy link
Contributor Author

forki commented Nov 14, 2017

Would love to get it in, but @dsyme is not a fan. So I'm closing it.

@forki forki closed this Nov 14, 2017
@KevinRansom KevinRansom reopened this Nov 14, 2017
@KevinRansom
Copy link
Contributor

@dsyme, merely for my education, could you clarify the reasons this should not be pursued.

Thank you,

Kevin

@dsyme
Copy link
Contributor

dsyme commented Nov 17, 2017

@KevinRansom With very, very few exceptions we don't hand implement optimizations on a library-function-by-library-function basis - we look for general approaches like improvements to inline that can cope with a whole swathe of cases. I am not at all keen to see case-by-case optimizations hardwired into the compiler and we will end up shipping a critical bugs if we go down this path

From the discussion of #1525

My feeling is that rewriting of combinations of library functions (of the kind in this PR) doesn't fit into the kinds of optimizations we do in the compiler, nor do we want to go there at this stage, except perhaps in an experimental branch, unless we have a really comprehensive approach to the problem. There are things the F# compiler does, there are things it doesn't do.

For example, if we did start doing rewrites of this kind, there are a very large number we'd like applied - there are probably 300 or more useful rewrites over FSharp.Core functions. Ideally we would do these using a general rewriting approach (no doubt inspired by Haskell - see @polytypic link above - Coq, HOL, Isabelle and many other rewriting systems). But that's also a big topic.

There are risks to every optimization, and we've previously shipped critical bugs in new optimizations. Correctness, completeness, predictability, robustness-under-change, "obviousness", "learnability" ("reducing what a reasonable user needs to know") are all reasons why I'm loathe to start adding new rewrites of combinations-of-library-functions into the compiler, especially in cases where the user can reasonably apply the optimization manually once a hotspot is identified.

So overall my feeling is that this is in the "things we don't do" category, unless we're iterating to a much more general solution to the topic, or to a particular sub-domain of optimization.

One exception is the introduction of state machines. However this is an algorithmically non-trivial operation that is extremely difficult to code by hand. Ideally that would be generalized to other computational structures (notably Async), and potentially to further combinations of sequence functions.

@matthid
Copy link
Contributor

matthid commented Nov 17, 2017

we look for general approaches

I think the general approach has already been suggested: Make the compiler extendible in that direction (ie. allow user defined optimizations). This way "FSharp.Core" is not "special" but everyone can do it for their own library as well.

@gusty
Copy link
Contributor

gusty commented Dec 31, 2017

I agree with the reasons @dsyme explained.

So, an alternative could be to be able to decorate some functions with attributes that allow to generalize these kind of optimizations.

I don't have a detailed proposal in mind at the moment, but for example something like a [<Pure>] attribute, combined with a [<IsMap>] attribute may allow the compiler to apply the functor law to rewrite on map for Seq, List, Array, Async applications of [<Pure>] decorated functions.

@dsyme
Copy link
Contributor

dsyme commented Jan 12, 2018

@forki I'll close this per my reasoning above

@dsyme dsyme closed this Jan 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants