Improvement for exportAll batch

[Thanh-Thanh]

Hi all,

This exportAll batch is supposed to be executed at 2 a.m each day.
It normally does a loop over all the datasets (having harvestedFrom is null).

In our case, this concerns all our datasets and so, it takes too much time to finish the job (~3 hours), without doing any exports.

We wonder if you think of an improuvement for this by limiting on the "entry" datasets of this batch?

Another one suggest : an atomic API for exporting an dataset ?

Thanks in advanced,

Thanh Thanh

[poikilotherm]

Related to #5345

[djbrooke]

@sekmiller @landreev any thoughts here? If this is the process, I'm wondering if there's some way to shorten this by only looking at those records that have been updated in the last 24 hours or something?