Improve reporting of ingest failures

[landreev]

These are the errors that appear when users mouse over the red triangles indicating ingest failures.

Some immediate improvements to make:

• duplicate these with notifications sent to the owner (possibly: keep the mouseover messages brief; send a more detailed message in the notification)
• add a "dismiss" checkbox, for the owner to be able to make the red mark disappear from the page;
• make the messages more detailed/provide more information. for example: instead of reporting the first unparseable line in a CSV file, report all (or first 10? - some limit would help). make sure we always provide some error message - even if it's "the file can't be parsed at all, ingest can't open the file, no further information is available".

[pdurbin]

There's also the fact that the icon used for ingest failure seems to give wrong impression to the user, as if they did something wrong, that Dataverse is angry with them. Here's a representative quote:

"That’s exactly what’s happening – the file uploads, then takes a very long time to go through the ingest process (assuming because it’s a large dataset), and eventually gives me a big angry red triangle that says “Tabular Data Ingest Failed”." -- https://help.hmdc.harvard.edu/Ticket/Display.html?id=215748#txn-4279663

Maybe we could come up with a friendlier icon that isn't angry and red. 😄

[landreev]

Well, it's red and angry specifically to get their attention. But once they mouse over, we can definitely do a better job explaining what happened; and, among other things, communicate more clearly that this is not a fatal condition, that the file will still be available to their users; and, perhaps, that it's "not their fault" too.

[oscardssmith]

Some of the proposed changes here would be quite easy, others much harder. Reporting all failures rather than the first will be very difficult, because it would require being able to parse past the broken bit, which in many cases may not work. We could make the error messages nicer though, although with the new parser (see #3767), all errors in the parser will (hopefully) be the user's fault for supplying a broken CSV. I have no idea how to send a notification, but that should be pretty easy.

[jggautier]

I like the idea of sending a more detailed or less technical message in the notification, which could include a link to a guide in the documentation (if one needs to be written after all of the great CSV ingest work that's been done so far).

[TaniaSchlatter]

There's a help request that came in through RT recently about a user instance of this issue. Researcher depositing data had not realized that their files uploaded, because they saw errors: https://help.hmdc.harvard.edu/Ticket/Display.html?id=252818

[pdurbin]

We made some improvements in this PR:

• improve messaging for ingest failures, file upload #8096 #8271

@adam3smith what do you think? Better? Needs more work? I see you have this down as a papercut. 😄

[adam3smith]

Yeah, I've been noticing the improved messages coming through -- I think in terms of messages, this is about as good as it's going to get, thanks! (Obviously it'd also be great to fix at least some of the causes of ingest failure, but that's on different issues)

[cmbz]

To focus on the most important features and bugs, we are closing issues created before 2020 (version 5.0) that are not new feature requests with the label 'Type: Feature'.

If you created this issue and you feel the team should revisit this decision, please reopen the issue and leave a comment.