Select random document button

[jstray]

Random document selection is good for:

1. are all the documents in this node really similar?
2. random sampling is useful for certain reporting and analysis tasks

[adamhooper]

This is very difficult. To select a random document in a 1M-doc document set, the client's doclist would need to scroll to the appropriate location. Unfortunately, the scrollbar will become useless at that stage. (It will be far too long.) Then we'd need to decide: what happens if the user scrolls up? We could be swamped with HTTP requests.

There are solutions; I'm just warning that it's a lot of work, both design-wise and implementation-wise.

[jstray]

Ok, but we don't have 1M docsets yet. What happens right now when if you try to scroll to the bottom of a 50k docset?

[adamhooper]

You can't.

You can only load 20 documents at a time. If each request takes 500ms (including all computation), that makes 40 docs per second, 2,400 docs per minute, and then ... well, asymptotic slowness as the DOM gets more and more memory-intensive. I predict it would take at least an hour on a fast computer to conduct this experiment.

Keep in mind: even Excel limits users to 65k rows. And Excel has zillions of speed enhancements that we shouldn't expect from our web browsers.

Again: there are solutions. They take lots of work, in design and implementation.

[thejonas]

Can the random button be like a tag, so the list is only 1 document long? We tend to solve most of our problems by using tags, so why not here?

[adamhooper]

I think we'd be better off figuring out what the UI should look like.

Here's another idea: we could let users sort the document list! They could sort by "title", "key words", "date" (when we have it), and ... wait for it ... "random". (I'm not convinced "random" should actually be random, because we'll want its effects to be reproducible. Perhaps sorting by md5 hash or some-such....)

I think that's the least invasive, design-wise; it certainly gives us a way forward.

[jstray]

I like this sort idea, but it's a pretty maximal solution, and it's not clear to me that the other types of sorts are urgently needed. Surely there must be some simpler way to present a random doc to the user.

[adamhooper]

In terms of both design cost and implementation cost, I think this is the cheapest solution. It also helps in the future.

[adamhooper]

We have sorting now. "Sort by random" should be easier to implement now.