[RLlib] RecSim Interest evolution environment should use custom video sampler: IEvVideoSampler due to only one cluster being used.

[gjoliver]

Why are these changes needed?

Interest evolution env should probably use IEV video sampler, instead of the utility model video sampler.
Docs returned from this sampler contains actual features, instead of utility indices.
This may help your slateq runs.

Related issue number

Checks

• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[sven1977]

Thanks for the fix @gjoliver !

[sven1977]

Waiting for LINT to pass.

[sven1977]

What's the difference and why did we need to change this? In the original RecSim repo, they use:
choice_model.MultinomialProportionalChoiceModel.

[gjoliver]

The difference is how to handle negative logits.

MultinomialLogitChoiceModel uses p(x) = exp(x) / Sum_{y in scores} exp(y), while
MultinomialProportionalChoiceModel uses p(x) = (x - min_normalizer) / sum(x - min_normalizer). You need to know the lower bound of your output logits before you can convert everything to be positive.
intuitively, these 2 should work similarly, like the more negative model output is, the less likely it will get clicked on.

[gjoliver]

I actually noticed another issue with RecSim.

IEvUserModel is hardcoded to use UtilityModelUserSampler:
https://github.com/google-research/recsim/blob/55e50e4be736d222ffe8c2477ed1981b40f91605/recsim/environments/interest_evolution.py#L491-L492

So I don't know if switching to IEvVideoSampler actually works or not ...