Skip to content

Add Global timeout for map/reduce queries#1766

Merged
garrensmith merged 2 commits intoapache:masterfrom
garrensmith:add-partition-timeout
Jan 21, 2019
Merged

Add Global timeout for map/reduce queries#1766
garrensmith merged 2 commits intoapache:masterfrom
garrensmith:add-partition-timeout

Conversation

@garrensmith
Copy link
Copy Markdown
Member

Overview

This makes the global timeout for a map/reduce and all_docs request
configurable via the config. It separates the config into global queries
and partition queries so that it is possible to make the global timeout
is for partitioned queries

Testing recommendations

Add the global_partition_view_timeout to the [fabric] config. You can then make queries and see if they timeout based on the value set.

Checklist

  • Code is written and works correctly;
  • Changes are covered by tests;
  • Documentation reflects the changes;

@davisp
Copy link
Copy Markdown
Member

davisp commented Nov 27, 2018

Looks good but obviously as currently written should wait till after we finish partition queries.

Copy link
Copy Markdown
Member

@davisp davisp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decided there's actually a few necessary tweaks but they're small.

Comment thread src/fabric/src/fabric_util.erl Outdated
global_view_timeout(Args) ->
PartitionQuery = couch_mrview_util:get_extra(Args, partition, false),
case PartitionQuery of
false -> timeout("global_view_timeout", "infinity");
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're doubling up the _timeout suffix cause the timeout/2 call appends that as well. Also global_view_timeout should probably be shortened to just view_timeout because you're looking at global vs partitioned. Also global_partition is a bit of a misnomer and should just be partition_view_timeout.

Also I think the SOP is to add commented out config options with their default values in default.ini. If the other timeout config options aren't already present I'd add all five settings in a separate commit on this PR.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed the global_ part for the names. The reason I added that is that rexi has two timeouts. A timeout for how long to wait for each response from a node and then an overall global timeout for the node to send all responses. The view and partition_view are for the second situation where as the all_docs and request timeouts are fo the individual requests.

@davisp davisp closed this Nov 27, 2018
@garrensmith garrensmith changed the base branch from feature/user-partitioned-databases-davisp to feature/database-partitions November 29, 2018 06:47
@garrensmith garrensmith reopened this Nov 29, 2018
@davisp
Copy link
Copy Markdown
Member

davisp commented Nov 29, 2018

Whoops. No idea how I managed to close this. Definitely wasn't on purpose.

Copy link
Copy Markdown
Member

@davisp davisp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last minor nit.

Comment thread src/fabric/src/fabric_util.erl Outdated
attachments_timeout() ->
timeout("attachments", "600000").

global_view_timeout(Args) ->
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still seems odd to be calling this global_view_timeout when it applies to both global and partitioned.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed the name

@davisp davisp force-pushed the feature/database-partitions branch 10 times, most recently from 4da3bab to fc696e8 Compare December 7, 2018 21:26
@davisp davisp force-pushed the feature/database-partitions branch 11 times, most recently from 90be3b0 to 6c2785a Compare December 19, 2018 17:51
@davisp davisp force-pushed the feature/database-partitions branch 2 times, most recently from 005b442 to 60bbe3f Compare December 20, 2018 20:48
@davisp davisp force-pushed the feature/database-partitions branch 5 times, most recently from be1fe52 to 31a4a53 Compare January 8, 2019 17:35
@davisp davisp force-pushed the feature/database-partitions branch 7 times, most recently from 04146c3 to 51a482e Compare January 16, 2019 23:21
@davisp davisp force-pushed the feature/database-partitions branch 2 times, most recently from 8cd68be to 91af772 Compare January 18, 2019 18:35
@davisp davisp closed this Jan 18, 2019
@garrensmith garrensmith changed the base branch from feature/database-partitions to master January 21, 2019 09:16
@garrensmith garrensmith reopened this Jan 21, 2019
This makes the global timeout for a map/reduce and all_docs request
configurable via the config. It separates the config into global queries
and partition queries so that it is possible to make the global timeout
is for partitioned queries
@davisp
Copy link
Copy Markdown
Member

davisp commented Jan 21, 2019

+1

@garrensmith garrensmith merged commit 33e3625 into apache:master Jan 21, 2019
@wohali wohali mentioned this pull request Feb 7, 2019
@garrensmith garrensmith deleted the add-partition-timeout branch December 12, 2019 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants