Internal Server Error on dataset page after deleting files, Vanishing files from already published versions.

[lmaylein]

Calling https://heidata.uni-heidelberg.de/dataset.xhtml?persistentId=doi:10.11588/data/DVU14P results in an Internal Server Error.
I have deleted files from this dataset (as I did many times before) to replace them in the resulting draft version and republish the dataset with the replaced files. Already when deleting the files I got an error message. Now the dataset is no longer accessible at all (in any version).

How can I make the dataset accessible again?

server.log:

[2022-07-29T11:47:34.701+0200] [Payara 5.2021.6] [SEVERE] [] [javax.enterprise.resource.webcontainer.jsf.application] [tid: _ThreadID=109 _ThreadName=http-thread-pool::jk-connector(4)] [timeMillis: 1659088054701] [levelValue: 1000] [[
  Error Rendering View[/dataset.xhtml]
javax.el.ELException: /dataset-citation.xhtml @16,76 rendered="#{!empty DatasetPage.thumbnailString and !DatasetPage.workingVersion.deaccessioned}": java.lang.NullPointerException
	at com.sun.faces.facelets.el.TagValueExpression.getValue(TagValueExpression.java:77)
	at javax.faces.component.ComponentStateHelper.eval(ComponentStateHelper.java:170)
	at javax.faces.component.UIComponentBase.isRendered(UIComponentBase.java:313)
	at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:271)
	at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:283)
	at com.sun.faces.renderkit.html_basic.PassthroughRenderer.encodeChildren(PassthroughRenderer.java:80)
	at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:566)
	at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1647)
	at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1650)
	at javax.faces.render.Renderer.encodeChildren(Renderer.java:152)
	at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:566)
	at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1647)
	at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1650)
	at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1650)
	at com.sun.faces.application.view.FaceletViewHandlingStrategy.renderView(FaceletViewHandlingStrategy.java:468)
	at com.sun.faces.application.view.MultiViewHandler.renderView(MultiViewHandler.java:170)
	at javax.faces.application.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:132)
	at javax.faces.application.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:132)
	at javax.faces.application.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:132)
	at org.omnifaces.viewhandler.OmniViewHandler.renderView(OmniViewHandler.java:121)
	at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:102)
	at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:76)
	at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:199)
	at javax.faces.webapp.FacesServlet.executeLifecyle(FacesServlet.java:708)
	at javax.faces.webapp.FacesServlet.service(FacesServlet.java:451)
	at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1636)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:331)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
	at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:282)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
	at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:226)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:257)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
	at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:757)
	at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:577)
	at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:158)
	at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:371)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:238)
	at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
	at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
	at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
	at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
	at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:218)
	at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
	at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
	at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
	at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: javax.el.ELException: java.lang.NullPointerException
	at javax.el.BeanELResolver.getValue(BeanELResolver.java:304)
	at com.sun.faces.el.DemuxCompositeELResolver._getValue(DemuxCompositeELResolver.java:156)
	at com.sun.faces.el.DemuxCompositeELResolver.getValue(DemuxCompositeELResolver.java:184)
	at com.sun.el.parser.AstValue.getValue(AstValue.java:114)
	at com.sun.el.parser.AstValue.getValue(AstValue.java:177)
	at com.sun.el.parser.AstEmpty.getValue(AstEmpty.java:42)
	at com.sun.el.parser.AstNot.getValue(AstNot.java:39)
	at com.sun.el.parser.AstAnd.getValue(AstAnd.java:34)
	at com.sun.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:184)
	at org.jboss.weld.module.web.el.WeldValueExpression.getValue(WeldValueExpression.java:50)
	at com.sun.faces.facelets.el.TagValueExpression.getValue(TagValueExpression.java:73)
	... 60 more
Caused by: java.lang.NullPointerException
	at edu.harvard.iq.dataverse.dataaccess.FileAccessIO.getDatasetDirectory(FileAccessIO.java:557)
	at edu.harvard.iq.dataverse.dataaccess.FileAccessIO.getFileSystemPath(FileAccessIO.java:443)
	at edu.harvard.iq.dataverse.dataaccess.ImageThumbConverter.generatePDFThumbnail(ImageThumbConverter.java:200)
	at edu.harvard.iq.dataverse.dataaccess.ImageThumbConverter.getImageThumbnailAsBase64(ImageThumbConverter.java:444)
	at edu.harvard.iq.dataverse.ThumbnailServiceWrapper.getAssignedDatasetImage(ThumbnailServiceWrapper.java:72)
	at edu.harvard.iq.dataverse.ThumbnailServiceWrapper.getDatasetCardImageAsBase64Url(ThumbnailServiceWrapper.java:252)
	at edu.harvard.iq.dataverse.ThumbnailServiceWrapper$Proxy$_$$_WeldClientProxy.getDatasetCardImageAsBase64Url(Unknown Source)
	at edu.harvard.iq.dataverse.DatasetPage.getThumbnailString(DatasetPage.java:467)
	at jdk.internal.reflect.GeneratedMethodAccessor879.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at javax.el.BeanELResolver.getValue(BeanELResolver.java:299)
	... 70 more
]]

[2022-07-29T11:47:34.703+0200] [Payara 5.2021.6] [WARNING] [] [javax.enterprise.web] [tid: _ThreadID=109 _ThreadName=http-thread-pool::jk-connector(4)] [timeMillis: 1659088054703] [levelValue: 900] [[
  StandardWrapperValve[Faces Servlet]: Servlet.service() for servlet Faces Servlet threw exception
java.lang.NullPointerException
	at edu.harvard.iq.dataverse.dataaccess.FileAccessIO.getDatasetDirectory(FileAccessIO.java:557)
	at edu.harvard.iq.dataverse.dataaccess.FileAccessIO.getFileSystemPath(FileAccessIO.java:443)
	at edu.harvard.iq.dataverse.dataaccess.ImageThumbConverter.generatePDFThumbnail(ImageThumbConverter.java:200)
	at edu.harvard.iq.dataverse.dataaccess.ImageThumbConverter.getImageThumbnailAsBase64(ImageThumbConverter.java:444)
	at edu.harvard.iq.dataverse.ThumbnailServiceWrapper.getAssignedDatasetImage(ThumbnailServiceWrapper.java:72)
	at edu.harvard.iq.dataverse.ThumbnailServiceWrapper.getDatasetCardImageAsBase64Url(ThumbnailServiceWrapper.java:252)
	at edu.harvard.iq.dataverse.ThumbnailServiceWrapper$Proxy$_$$_WeldClientProxy.getDatasetCardImageAsBase64Url(Unknown Source)
	at edu.harvard.iq.dataverse.DatasetPage.getThumbnailString(DatasetPage.java:467)
	at jdk.internal.reflect.GeneratedMethodAccessor879.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at javax.el.BeanELResolver.getValue(BeanELResolver.java:299)
	at com.sun.faces.el.DemuxCompositeELResolver._getValue(DemuxCompositeELResolver.java:156)
	at com.sun.faces.el.DemuxCompositeELResolver.getValue(DemuxCompositeELResolver.java:184)
	at com.sun.el.parser.AstValue.getValue(AstValue.java:114)
	at com.sun.el.parser.AstValue.getValue(AstValue.java:177)
	at com.sun.el.parser.AstEmpty.getValue(AstEmpty.java:42)
	at com.sun.el.parser.AstNot.getValue(AstNot.java:39)
	at com.sun.el.parser.AstAnd.getValue(AstAnd.java:34)
	at com.sun.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:184)
	at org.jboss.weld.module.web.el.WeldValueExpression.getValue(WeldValueExpression.java:50)
	at com.sun.faces.facelets.el.TagValueExpression.getValue(TagValueExpression.java:73)
	at javax.faces.component.ComponentStateHelper.eval(ComponentStateHelper.java:170)
	at javax.faces.component.UIComponentBase.isRendered(UIComponentBase.java:313)
	at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:271)
	at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:283)
	at com.sun.faces.renderkit.html_basic.PassthroughRenderer.encodeChildren(PassthroughRenderer.java:80)
	at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:566)
	at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1647)
	at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1650)
	at javax.faces.render.Renderer.encodeChildren(Renderer.java:152)
	at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:566)
	at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1647)
	at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1650)
	at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1650)
	at com.sun.faces.application.view.FaceletViewHandlingStrategy.renderView(FaceletViewHandlingStrategy.java:468)
	at com.sun.faces.application.view.MultiViewHandler.renderView(MultiViewHandler.java:170)
	at javax.faces.application.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:132)
	at javax.faces.application.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:132)
	at javax.faces.application.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:132)
	at org.omnifaces.viewhandler.OmniViewHandler.renderView(OmniViewHandler.java:121)
	at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:102)
	at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:76)
	at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:199)
	at javax.faces.webapp.FacesServlet.executeLifecyle(FacesServlet.java:708)
	at javax.faces.webapp.FacesServlet.service(FacesServlet.java:451)
	at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1636)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:331)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
	at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:282)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
	at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:226)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:257)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
	at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:757)
	at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:577)
	at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:158)
	at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:371)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:238)
	at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
	at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
	at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
	at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
	at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:218)
	at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
	at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
	at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
	at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
	at java.base/java.lang.Thread.run(Thread.java:829)
]]

[lmaylein]

I have now commented out the corresponding img tag in dataset-citation.xhtml. This allows me to display the dataset. In doing so, I found that the files were indeed deleted, but directly in the published version V8. No draft version was created automatically :-(

[lmaylein]

Correction: There is a DRAFT version but the files have been deleted in the published version V 8.0
(e.g. einfuehrungstexte.pdf)

[lmaylein]

For the six files in the screenshot above, it is noticeable that the owner_id in dvobject is empty. In the other files of this dataset the owner_id is assigned.

[landreev]

We have spent some time trying to diagnose this issue today; this appears to be a very serious problem, and we are working to produce a fix asap.

For the six files in the screenshot above, it is noticeable that the owner_id in dvobject is empty. In the other files of this dataset the owner_id is assigned.

Could you please clarify what's going on. So, the dvobjects are still there - but the owner_id got nulled. This is consistent with what we are seeing in the code that was added in 5.11. That can of course be fixed - by setting owner_id=[dataset id].
Also, in the screen shot you posted, the files seem to actually appear in the published version (8.0), at least in the version diff view. Does that mean that the filemetadata objects are still intact in the database for these 6 files in version 8.0?

Unfortunately, in addition to these problems on the database side, there is also a chance that the physical files for these 6 datafiles have been erased, from the filesystem/S3, whichever you are using. Could you please check your storage and confirm this? This of course would be the worst part, so this is why we are trying to fix this issue asap.

[landreev]

Hello,
This is a very crudely-patched dataverse-5.11 war file I just deployed on our production servers: http://dataverse.harvard.edu/patch/dataverse-5.11.war
It may not be fixing the bug in question fully/properly, but it should prevent another user from erasing their files unintentionally. We will produce a finalized patch on Monday.
Our apologies for this!

[lmaylein]

This is, how it looks like in dvobject:

dvndb=# select * from dvobject where dtype = 'DataFile' and identifier like '%DVU14P%' and owner_id is null;
  id  |  dtype   |       createdate        | indextime |    modificationtime     | permissionindextime | permissionmodificat
iontime | previewimageavailable |     publicationdate     | creator_id | owner_id | releaseuser_id |        storageidentifie
r        | authority |   globalidcreatetime    | identifierregistered |     identifier     | protocol 
------+----------+-------------------------+-----------+-------------------------+---------------------+--------------------
--------+-----------------------+-------------------------+------------+----------+----------------+------------------------
---------+-----------+-------------------------+----------------------+--------------------+----------
 6855 | DataFile | 2022-05-03 08:47:20.036 |           | 2022-07-29 11:42:24.024 |                     | 2022-05-03 08:47:08
.128    | t                     | 2022-05-03 08:57:05.221 |         10 |          |                | file://18088ac3260-53d7
4005097e | 10.11588  | 2022-05-03 08:57:05.221 | t                    | data/DVU14P/X2AZYE | doi
 6858 | DataFile | 2022-05-03 08:48:03.858 |           | 2022-07-29 11:42:24.024 |                     | 2022-05-03 08:47:58
.954    | f                     | 2022-05-03 08:57:05.221 |         10 |          |                | file://18088acf8ea-4c5d
39089afc | 10.11588  | 2022-05-03 08:57:05.221 | t                    | data/DVU14P/0RIECM | doi
 5843 | DataFile | 2022-01-07 08:50:37.618 |           | 2022-07-29 11:42:24.024 |                     | 2022-01-07 08:50:30
.81     | f                     | 2022-01-07 09:03:59.325 |         10 |          |                | file://17e3384889a-94f3
0848d41f | 10.11588  | 2022-05-03 08:57:05.221 | t                    | data/DVU14P/XN0EZS | doi
 5845 | DataFile | 2022-01-07 08:51:07.753 |           | 2022-07-29 11:42:24.024 |                     | 2022-01-07 08:51:05
.416    | t                     | 2022-01-07 09:03:59.325 |         10 |          |                | file://17e33850fc8-c1f6
057541e5 | 10.11588  | 2022-05-03 08:57:05.221 | t                    | data/DVU14P/UX2UVB | doi
 6859 | DataFile | 2022-05-03 08:48:03.858 |           | 2022-07-29 11:42:24.024 |                     | 2022-05-03 08:47:59
.131    | f                     | 2022-05-03 08:57:05.221 |         10 |          |                | file://18088acf99b-8500
47840b2d | 10.11588  | 2022-05-03 08:57:05.221 | t                    | data/DVU14P/MEBHOF | doi
 6860 | DataFile | 2022-05-03 08:48:30.056 |           | 2022-07-29 11:42:24.024 |                     | 2022-05-03 08:48:27
.987    | f                     | 2022-05-03 08:57:05.221 |         10 |          |                | file://18088ad6a53-dad9
3b4a9ad3 | 10.11588  | 2022-05-03 08:57:05.221 | t                    | data/DVU14P/RRAQ2H | doi
(6 rows)

dvndb=# select * from filemetadata where datafile_id = 6860;
  id  | description | directorylabel |        label         | restricted | version | datafile_id | datasetversion_id | prov_
freeform 
------+-------------+----------------+----------------------+------------+---------+-------------+-------------------+------
---------
 8493 |             |                | eichstaett1611.xhtml | f          |       1 |        6860 |               794 | 
(1 row)

And yes, it seems that the files were erased in files/studies/10.11588/data/DVU14P.

[lmaylein]

Can I get the files back from backup and set the owner_id in dvobjects manually to possibly fix the problem or should I keep the current state for further debugging?

[landreev]

Yes, I believe resetting the owner_id in these dvobjects, and restoring the physical files from backups will fix this.
Please let us know if this works. Otherwise, it looks like we now know what's going on. The symptoms on your end are consistent with the bug we have found (it's literally one line of code that was moved into a wrong place by accident in 5.11).

[lmaylein]

Yes, I believe resetting the owner_id in these dvobjects, and restoring the physical files from backups will fix this.
Please let us know if this works.

Yes, it works. I was also able to undo the changes in dataset-citation.xhtml.

Thanks.

[landreev]

Glad to hear it worked.
I noticed that you are still running stock 5.11. I don't know how your installation operates, if you have data curating users who are not you. But if there is even a slightest chance that somebody else may attempt to delete files from a published version again, I would strongly recommend to deploy my temporary patch asap. (although the official patch release 5.11.1 should be out in the morning, our time).

This is a seriously destructive bug (yours was not the worst case). Sorry you had to be the first known victim of it, but thank you for bringing it to our attention.