Use PHP generators to get around some memory limitations

[schlessera]

We should investigate the possibility to use PHP generators instead of normal loops for some of the commands that tend to run into memory issues on larger sites.

PHP generators have only been included with PHP 5.5+, so we'd need to provide conditionally loaded implementations for these. However, I think this might make sense for a selection of commands.

[swissspidy]

Some wp * list commands in the entity-command package fetch all items (users, posts, terms, comments, sites) into an array before processing and formatting them. On large sites, fetching thousands of objects into memory simultaneously could cause memory issues. So those commands might be good candidates.

We'd need to test whether this works well with Formatter. For the table format (which needs to calculate column widths), Formatter might buffer the generator anyway.

Also, ideally we have some way to verify memory usage before and after this change.

[swissspidy]

Related:

• Make large queries scalable #91