TDL/7493 Improve BagGenerator failure handling

[qqmyers]

What this PR does / why we need it: The prior code did not correctly close connections when a non-200 response was received. Due to this, repeated failures could temporarily exhaust the pool of threads. While this was first seen with 'bad' datasets, e.g. ones where the physical files have been deleted, it would happen if there were temporary network outages as well. The PR correctly closes the stream in those cases.

It also adds a couple minor fixes/enhancements:

• Calls to the Dataverse API when using an AWS load balancer can generate 'Invalid Cookie Header' warnings. While it is possible that most of the ones seen at TDL come from the AWS library in the S3 Store, the recommended fix is to accept Standard cookies (which the aws library apparently doesn't do), so that is added here for the Apache Client calls.
• The Bag spec requires a manifest file which we weren't generating for datasets with no files. To be slightly more compliant with that, this PR creates an empty manifest and updates the log message to make it clearer that this is not necessarily an error (i.e. this happens normally if a dataset has no files.).
• If a 404 error is received, the code will no longer attempt to retry. If a >=500 error, such as gateway timeouts are received, the code will still attempt up to 5 retries.

Which issue(s) this PR closes:

Closes #8603

Special notes for your reviewer:

Suggestions on how to test this:

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

Is there a release notes update needed for this change?: planning an aggregate Bag/archiving change notes rather than per PR.

Additional documentation:

[coveralls]

Coverage decreased (-0.004%) to 18.998% when pulling a9f3aa5 on TexasDigitalLibrary:TDL/7493-improve_BagGnerator_failure_handling into 4e991dd on IQSS:develop.