Skip to content

EC2 autoscaler: avoid hitting aws filter limits#1954

Merged
drcrallen merged 1 commit intoapache:masterfrom
metamx:fix-stupid-aws-limit
Nov 13, 2015
Merged

EC2 autoscaler: avoid hitting aws filter limits#1954
drcrallen merged 1 commit intoapache:masterfrom
metamx:fix-stupid-aws-limit

Conversation

@xvrl
Copy link
Copy Markdown
Member

@xvrl xvrl commented Nov 11, 2015

Prevents the following error below from happening.
As a side benefit this also materialized the instanceId list, so it should reduce garbage creation.

com.amazonaws.AmazonServiceException: The maximum number of filter values specified on a single call is 200 (Service: AmazonEC2; Status Code: 400; Error Code: FilterLimitExceeded; Request ID: XXXXX)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this documented by AWS somewhere? if so can we put a comment where this came from?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't searched the docs, but the limit appears to be 200 by default.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we also add a comment ?

@drcrallen
Copy link
Copy Markdown
Contributor

👍

1 similar comment
@nishantmonu51
Copy link
Copy Markdown
Member

👍

@xvrl
Copy link
Copy Markdown
Member Author

xvrl commented Nov 11, 2015

hold on before merging, there's another place where we might hit this limit

@xvrl xvrl force-pushed the fix-stupid-aws-limit branch from 0a80ba2 to d42c8b3 Compare November 11, 2015 22:27
@drcrallen
Copy link
Copy Markdown
Contributor

Still 👍

@xvrl
Copy link
Copy Markdown
Member Author

xvrl commented Nov 11, 2015

ok I added comment, and fix the other filter query as well, please have a look

@drcrallen
Copy link
Copy Markdown
Contributor

I'm guessing there's no way to check this in a unit test is there?

@xvrl
Copy link
Copy Markdown
Member Author

xvrl commented Nov 12, 2015

@drcrallen I can add a unit test, sure,

@xvrl xvrl force-pushed the fix-stupid-aws-limit branch from d42c8b3 to 065dfd0 Compare November 12, 2015 04:24
@xvrl xvrl force-pushed the fix-stupid-aws-limit branch from 065dfd0 to 749ac12 Compare November 12, 2015 04:28
@xvrl
Copy link
Copy Markdown
Member Author

xvrl commented Nov 12, 2015

added tests

@nishantmonu51
Copy link
Copy Markdown
Member

👍

@xvrl xvrl added this to the 0.8.3 milestone Nov 12, 2015
drcrallen added a commit that referenced this pull request Nov 13, 2015
EC2 autoscaler: avoid hitting aws filter limits
@drcrallen drcrallen merged commit 46527a9 into apache:master Nov 13, 2015
@drcrallen drcrallen deleted the fix-stupid-aws-limit branch November 13, 2015 18:52
@xvrl xvrl mentioned this pull request Dec 1, 2015
@gianm gianm mentioned this pull request Dec 4, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants