Requesting support for a newer version of Counter Processor

[agostinho-gomes]

Hi
our version of Dataverse is 4.20 .
a few days ago, we decide to use MAKE DATA COUNT for metrics/statistics.
so we download last version of "counter processor" - https://github.com/CDLUC3/counter-processor, via command "git" on Linux Ubuntu.
After Install and configure it, as in documentation of Dataverse.
After process counter logs, a report json was created.
when we try to upload json on dataverse, we receive this error:
"curl -X POST "http://localhost:8080/api/admin/makeDataCount/addUsageMetricsFromSushiReport?reportOnDisk=/counter-processor/reports/mdc-report.json"
{"status":"ERROR","code":500,"message":"Internal server error. More details available at the server logs.","incidentId":"a4a2eafa-8a47-490e-84dd-45564acfd388"}"

file: server.log
_[2022-01-31T11:49:04.060+0000] [glassfish 4.1] [SEVERE] [] [edu.harvard.iq.dataverse.api.errorhandlers.ServeletExceptionHandler] [tid: _ThreadID=28 ThreadName=http-listener-1(3)] [timeMillis: 1643629744060] [levelValue: 1000] [[
API internal error a4a2eafa-8a47-490e-84dd-45564acfd388: Null Pointer
java.lang.NullPointerException
at edu.harvard.iq.dataverse.makedatacount.DatasetMetricsServiceBean.parseSushiReport(DatasetMetricsServiceBean.java:122)

link to "DatasetMetricsServiceBean.java" line 122
https://github.com/IQSS/dataverse/blob/v4.20/src/main/java/edu/harvard/iq/dataverse/makedatacount/DatasetMetricsServiceBean.java#L122

seems that Dateverse is not compatible with last version of counter-processor.
the json file created with last verion of counter-processor has "report-datasets" with a hyphen/dash (-) and the Dataverse code expects report_datasets with an underscore (_).

after that,
we uninstall this version of counter-processor.
install version that is on documentation of Dataverse.
and we make upload of json to Dataverse with no problems.

curl -X POST "http://localhost:8080/api/admin/makeDataCount/addUsageMetricsFromSushiReport?reportOnDisk=/usr/local/counter-processor-0.0.1/reports/mdc-report.json"
{"status":"OK","data":{"message":"Usage Metrics Data has been added to all datasets from file /usr/local/counter-processor-0.0.1/reports/mdc-report.json"}}

best regard
Agostinho Gomes
Minho University
https://datarepositorium.uminho.pt/

[qqmyers]

Thanks for the issue! FWIW: I think the MDC work started when 0.0.1 was the only released version. It makes sense to update though.

Any further info you might have on how the reports differ would help in scoping this work. If the only change is from a dash to underscore in the name, it would be simple, but any change to the report structure (I see the release notes talk about adding sizes) would add work. (If you manually rename the file is it parsed correctly by Dataverse?) We'll also have to update the documentation about counter and its prerequisites as well.

[agostinho-gomes]

counter-processor_reports.zip
this zip file contains 2 reports, from processs-counter, one from version 0.0.1 and other from last version of couter-processor.

[qqmyers]

So far all I'm seeing is the one character change you noted, a new warning about using gzip (seems like we can ignore), and a change in the order of entries (also shouldn't matter). If that's all this should be pretty trivial to fix (and/or to accept both variants). If you notice anything else, let us know.