Search API: Support search without an API Token

[michbarsinai]

Search should be as if used by :guest.

[michbarsinai]

related: #1299

[michbarsinai]

Also related: #1838

[michbarsinai]

Implemented, ready for QA.

[pdurbin]

Implemented, ready for QA.

@michbarsinai what about code review?

[pdurbin]

To me this goes all the way back to #1809 (comment) and reverses a policy decision made by @mercecrosas for the out-of-the-box behavior of Dataverse.

Also, it seems like all the logic surrounding :SearchApiNonPublicAllowed has been circumvented. In #1299 is was decided that we're only supporting public search.

In short, there are a lot of policy changes in pull request #3908, a lot of implications to unpack. I am very supportive of work toward "Consider options for opening APIs without tokens" in #1838 but I feel that we need to make sure whatever changes happen are what we want as a project.

[pdurbin]

In standup I suggested that @djbrooke could ping @mercecrosas about the policy considerations. Also, I forgot to mention something I said to @michbarsinai which is that if we open up a lot of APIs (#1838) we should consider rate limiting the API (#1339) like GitHub and others. Otherwise, you're down at a low level blocking individual abusive IP addresses with iptables or whatever.

[djbrooke]

Talked with @mercecrosas and @scolapasta about this yesterday. We should go ahead with this change. If so many people are using our Search API that we have load issues or need to IP ban a few bad apples, we'll revisit or determine some other strategy to handle this.