test failure in develop: HarvestingServerIT.testOaiFunctionality

[pdurbin]

What steps does it take to reproduce the issue?

• Open HarvestingServerIT and run testOaiFunctionality.
• Observe that the test fails

The output from the test is something like the XML below.

<OAI-PMH schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
  <responseDate>2022-08-25T14:00:57Z</responseDate>
  <request verb="ListIdentifiers" metadataPrefix="oai_dc" set="YZOIGQ">http://localhost:8080/oai</request>
  <ListIdentifiers>
    <header>
      <identifier>doi:10.5072/FK2/YZOIGQ</identifier>
      <datestamp>2022-08-25T14:00:52Z</datestamp>
      <setSpec>YZOIGQ</setSpec>
    </header>
  </ListIdentifiers>
</OAI-PMH>

The test is looking for a list of size 1:

    // Run ListIdentifiers on this newly-created set:
    Response listIdentifiersResponse = UtilIT.getOaiListIdentifiers(setName, "oai_dc");
    List ret = listIdentifiersResponse.getBody().xmlPath().getList("OAI-PMH.ListIdentifiers.header");

    assertEquals(OK.getStatusCode(), listIdentifiersResponse.getStatusCode());
    assertNotNull(ret);
    logger.info("setName: " + setName);
    logger.info("listIdentifiersResponse.prettyPrint:..... ");
    // There should be 1 and only 1 record in the response:
    assertEquals(1, ret.size());

Which version of Dataverse are you using?

The first observed failure on Jenkins on the develop branch was for 1ef73b0 (the tip of develop as of this writing).

We're also seeing the failure in PR #8880 where the tip of develop was merged.

Locally I'm on 6ca1c9d (I haven't pulled from develop so I haven't picked up the merge from PR #8901 yet) and @sekmiller also hasn't pulled and we are both seeing the failure.

[landreev]

Quick status update - after much poking/discussing it on slack, it appears to be a) not obviously connected to any recent changes and b) intermittent. And seems like another instance of the overall test suite becoming more resource-intensive and slowing down to the point where an individual test that relies on a hard-coded sleep to ensure the completion of previous steps starts failing intermittently.
In this specific case there's a 5 sec. sleep in this test meant to let the OAI set re-export to finish. I'll make a PR doubling that interval to 10 sec.

Granted, there are smarter ways to go about it; but not sure it’s worth it at this point. I have opened the issue #8843 recently for expanding the number and scope of our Harvesting/OAI tests that I very much want to prioritize. Anything “smarter” should be done as part of that effort.

Edit: actually, the test was failing not on account of the set create taking too long, but because we were trying to create the set before the publish command had a chance to complete on the dataset there. Or, to be more specific, while the dataset was still being indexed and imported - the tasks that the publish command executes asynchronously. So it wasn't the 5 sec. sleep that needed to be increased; it was another short sleep that needed to be added before the set create step.