Optional tabular ingest skipping

[lubitchv]

We have a pilot project of transitioning from Nesstar to Dataverse. One of the features of Nesstar is the ability to convert files into different formats on the fly, such as SPSS, STATA etc. Dataverse does not have this feature, but our users really would like to have all these different formats. To get around this we plan to import all these formats from Nesstar. The problem is all these files will go through tabular ingest, but we would like that only one (SPSS) file would go through tabular ingest but other files in other formats such as STATA would be added to datasets without ingest.

Therefore it would be nice to have an optional parameter (tabularIngest=true/false) in add file to dataset API. Maybe this parameter would be passed down to saveAndAddFilesToDataset function in IngestServiceBean and maybe something like

if (FileUtil.canIngestAsTabular(dataFile))
replaced with
if (FileUtil.canIngestAsTabular(dataFile) && tabularIngest)

What would be the thoughts regarding it?

[qqmyers]

Have you considered the aux file API? If you add your alternate formats that way they appear in the download menu but would not be separate, first-class datafiles in the file list and wouldn't undergo ingest, etc. FWIW: We're doing ~this at QDR where we're converting docx to pdf and storing the pdf as an aux file.

[landreev]

Uploading alternative formats as auxiliary files may be a perfectly good solution for this use case. But I would very much like to finally implement this "skip option" flag/parameter, both for the API and for the UI, regardless.

[lubitchv]

We will need such feature for add file API soon. I can make a pull request for add API with parameter with existing code, if it is ok with everyone. Otherwise I will look into auxiliary files or add/uningest and will discuss it with my group.