-
-
Notifications
You must be signed in to change notification settings - Fork 753
Fix Issue 18096 - Add fold() to std.parallelism #5951
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
0074d24
Fix a couple of tuple links
acehreli c472b23
Fix Issue 18096 - Add fold() to std.parallelism
acehreli 673c4e9
Apply review comments: Backticks for inline code, StdUnittest, etc.
acehreli 665e4ad
Implement std.parallelism.fold with static if branches instead of wit…
acehreli 7519d93
Add the changelog entry
acehreli 8cfff5e
Use message with static assert(0), use aliases instead of nested func…
acehreli fec204e
Merge branch 'master' of github.com:dlang/phobos into TaskPool_fold
acehreli f9e3dde
Use string lambdas in non-documentation unittests to reduce memory us…
acehreli c1c8e86
remove surprising emtpy line
andralex File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| `fold` is added to `std.parallelism.TaskPool` | ||
|
|
||
| $(REF fold, std, parallelism, TaskPool) is functionally equivalent to | ||
| $(REF_ALTTEXT `reduce`, reduce, std, parallelism, TaskPool) except the range | ||
| parameter comes first and there is no need to use $(REF_ALTTEXT | ||
| `tuple`,tuple,std,typecons) for multiple seeds. | ||
|
|
||
| --- | ||
| static int adder(int a, int b) | ||
| { | ||
| return a + b; | ||
| } | ||
| static int multiplier(int a, int b) | ||
| { | ||
| return a * b; | ||
| } | ||
|
|
||
| // Just the range | ||
| auto x = taskPool.fold!adder([1, 2, 3, 4]); | ||
| assert(x == 10); | ||
|
|
||
| // The range and the seeds (0 and 1 below; also note multiple | ||
| // functions in this example) | ||
| auto y = taskPool.fold!(adder, multiplier)([1, 2, 3, 4], 0, 1); | ||
| assert(y[0] == 10); | ||
| assert(y[1] == 24); | ||
|
|
||
| // The range, the seed (0), and the work unit size (20) | ||
| auto z = taskPool.fold!adder([1, 2, 3, 4], 0, 20); | ||
| assert(z == 10); | ||
| --- |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2421,12 +2421,15 @@ public: | |
| { | ||
| /** | ||
| Parallel reduce on a random access range. Except as otherwise noted, | ||
| usage is similar to $(REF _reduce, std,algorithm,iteration). This | ||
| function works by splitting the range to be reduced into work units, | ||
| which are slices to be reduced in parallel. Once the results from all | ||
| work units are computed, a final serial reduction is performed on these | ||
| results to compute the final answer. Therefore, care must be taken to | ||
| choose the seed value appropriately. | ||
| usage is similar to $(REF _reduce, std,algorithm,iteration). There is | ||
| also $(LREF fold) which does the same thing with a different parameter | ||
| order. | ||
|
|
||
| This function works by splitting the range to be reduced into work | ||
| units, which are slices to be reduced in parallel. Once the results | ||
| from all work units are computed, a final serial reduction is performed | ||
| on these results to compute the final answer. Therefore, care must be | ||
| taken to choose the seed value appropriately. | ||
|
|
||
| Because the reduction is being performed in parallel, $(D functions) | ||
| must be associative. For notational simplicity, let # be an | ||
|
|
@@ -2502,6 +2505,12 @@ public: | |
| After this function is finished executing, any exceptions thrown | ||
| are chained together via $(D Throwable.next) and rethrown. The chaining | ||
| order is non-deterministic. | ||
|
|
||
| See_Also: | ||
|
|
||
| $(LREF fold) is functionally equivalent to $(LREF _reduce) except the | ||
| range parameter comes first and there is no need to use | ||
| $(REF_ALTTEXT $(D tuple),tuple,std,typecons) for multiple seeds. | ||
| */ | ||
| auto reduce(Args...)(Args args) | ||
| { | ||
|
|
@@ -2803,6 +2812,128 @@ public: | |
| } | ||
| } | ||
|
|
||
| /// | ||
| template fold(functions...) | ||
| { | ||
| /** Implements the homonym function (also known as `accumulate`, `compress`, | ||
| `inject`, or `foldl`) present in various programming languages of | ||
| functional flavor. | ||
|
|
||
| `fold` is functionally equivalent to $(LREF reduce) except the range | ||
| parameter comes first and there is no need to use $(REF_ALTTEXT | ||
| `tuple`,tuple,std,typecons) for multiple seeds. | ||
|
|
||
| There may be one or more callable entities (`functions` argument) to | ||
| apply. | ||
|
|
||
| Params: | ||
| args = Just the range to _fold over; or the range and one seed | ||
| per function; or the range, one seed per function, and | ||
| the work unit size | ||
|
|
||
| Returns: | ||
| The accumulated result as a single value for single function and | ||
| as a tuple of values for multiple functions | ||
|
|
||
| See_Also: | ||
| Similar to $(REF _fold, std,algorithm,iteration), `fold` is a wrapper around $(LREF reduce). | ||
|
|
||
| Example: | ||
| --- | ||
| static int adder(int a, int b) | ||
| { | ||
| return a + b; | ||
| } | ||
| static int multiplier(int a, int b) | ||
| { | ||
| return a * b; | ||
| } | ||
|
|
||
| // Just the range | ||
| auto x = taskPool.fold!adder([1, 2, 3, 4]); | ||
| assert(x == 10); | ||
|
|
||
| // The range and the seeds (0 and 1 below; also note multiple | ||
| // functions in this example) | ||
| auto y = taskPool.fold!(adder, multiplier)([1, 2, 3, 4], 0, 1); | ||
| assert(y[0] == 10); | ||
| assert(y[1] == 24); | ||
|
|
||
| // The range, the seed (0), and the work unit size (20) | ||
| auto z = taskPool.fold!adder([1, 2, 3, 4], 0, 20); | ||
| assert(z == 10); | ||
| --- | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Obviously not a huge fan of this either, but fixing this is independent to this addition as |
||
| */ | ||
| auto fold(Args...)(Args args) | ||
| { | ||
| static assert(isInputRange!(Args[0]), "First argument must be an InputRange"); | ||
|
|
||
| alias range = args[0]; | ||
|
|
||
| static if (Args.length == 1) | ||
| { | ||
| // Just the range | ||
| return reduce!functions(range); | ||
| } | ||
| else static if (Args.length == 1 + functions.length || | ||
| Args.length == 1 + functions.length + 1) | ||
| { | ||
| static if (functions.length == 1) | ||
| { | ||
| alias seeds = args[1]; | ||
| } | ||
| else | ||
| { | ||
| auto seeds() | ||
| { | ||
| import std.typecons : tuple; | ||
| return tuple(args[1 .. functions.length+1]); | ||
| } | ||
| } | ||
|
|
||
| static if (Args.length == 1 + functions.length) | ||
| { | ||
| // The range and the seeds | ||
| return reduce!functions(seeds, range); | ||
| } | ||
| else static if (Args.length == 1 + functions.length + 1) | ||
| { | ||
| // The range, the seeds, and the work unit size | ||
| static assert(isIntegral!(Args[$-1]), "Work unit size must be an integral type"); | ||
| return reduce!functions(seeds, range, args[$-1]); | ||
| } | ||
| } | ||
| else | ||
| { | ||
| import std.conv : text; | ||
| static assert(0, "Invalid number of arguments (" ~ Args.length.text ~ "): Should be an input range, " | ||
| ~ functions.length.text ~ " optional seed(s), and an optional work unit size."); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // This test is not included in the documentation because even though these | ||
| // examples are for the inner fold() template, with their current location, | ||
| // they would appear under the outer one. (We can't move this inside the | ||
| // outer fold() template because then dmd runs out of memory possibly due to | ||
| // recursive template instantiation, which is surprisingly not caught.) | ||
| @system unittest | ||
| { | ||
| // Just the range | ||
| auto x = taskPool.fold!"a + b"([1, 2, 3, 4]); | ||
| assert(x == 10); | ||
|
|
||
| // The range and the seeds (0 and 1 below; also note multiple | ||
| // functions in this example) | ||
| auto y = taskPool.fold!("a + b", "a * b")([1, 2, 3, 4], 0, 1); | ||
| assert(y[0] == 10); | ||
| assert(y[1] == 24); | ||
|
|
||
| // The range, the seed (0), and the work unit size (20) | ||
| auto z = taskPool.fold!"a + b"([1, 2, 3, 4], 0, 20); | ||
| assert(z == 10); | ||
| } | ||
|
|
||
| /** | ||
| Gets the index of the current thread relative to this $(D TaskPool). Any | ||
| thread not in this pool will receive an index of 0. The worker threads in | ||
|
|
@@ -3335,6 +3466,28 @@ public: | |
| } | ||
| } | ||
|
|
||
| @system unittest | ||
| { | ||
| import std.algorithm.iteration : sum; | ||
| import std.range : iota; | ||
| import std.typecons : tuple; | ||
|
|
||
| enum N = 100; | ||
| auto r = iota(1, N + 1); | ||
| const expected = r.sum(); | ||
|
|
||
| // Just the range | ||
| assert(taskPool.fold!"a + b"(r) == expected); | ||
|
|
||
| // Range and seeds | ||
| assert(taskPool.fold!"a + b"(r, 0) == expected); | ||
| assert(taskPool.fold!("a + b", "a + b")(r, 0, 0) == tuple(expected, expected)); | ||
|
|
||
| // Range, seeds, and work unit size | ||
| assert(taskPool.fold!"a + b"(r, 0, 42) == expected); | ||
| assert(taskPool.fold!("a + b", "a + b")(r, 0, 0, 42) == tuple(expected, expected)); | ||
| } | ||
|
|
||
| /** | ||
| Returns a lazily initialized global instantiation of $(D TaskPool). | ||
| This function can safely be called concurrently from multiple non-worker | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Refs automatically use code boxes. If they don't we should fix it for all of them.
Also Phobos prefers backticks nowadays.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the review. Yeah, the font was regular text font without $(D). I'll replace with backticks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thinking more about this, none of the _ALTTEXT macros should use code boxes as the alternative text can be any free form text. I've just checked dlang.org.ddoc and these macros look correct to me. So, back to backticks...