[SPARK-17671] [WEBUI] Spark 2.0 history server summary page is slow even set spark.history.ui.maxApplications

[srowen]

What changes were proposed in this pull request?

Return Iterator of applications internally in history server, for consistency and performance. See #15248 for some back-story.

The code called by and calling HistoryServer.getApplicationList wants an Iterator, but this method materializes an Iterable, which potentially causes a performance problem. It's simpler too to make this internal method also pass through an Iterator.

How was this patch tested?

Existing tests.

[srowen]

CC @wgtmac and @ajbozarth for a look, in case I'm missing something.

[SparkQA]

Test build #66230 has finished for PR 15321 at commit 14fb9a0.

• This patch fails to build.
• This patch merges cleanly.
• This patch adds no public classes.

[wgtmac]

LGTM

[ajbozarth]

Looks like some tests need to be updated, but what's here LGTM

[SparkQA]

Test build #66242 has finished for PR 15321 at commit d90c38c.

• This patch fails MiMa tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #66243 has finished for PR 15321 at commit 315407c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[ajbozarth]

LGTM

[SparkQA]

Test build #66245 has finished for PR 15321 at commit e385be8.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

Merged to master