API Search enhancement - Returning metadatafields

[FNI18300]

Currently , edu.harvard.iq.dataverse.api.Search.search method takes (master branch) the following parameters :

@QueryParam("q") String query,
@QueryParam("type") final List<String> types,
@QueryParam("subtree") final List<String> subtrees,
@QueryParam("sort") String sortField,
@QueryParam("order") String sortOrder,
@QueryParam("per_page") final int numResultsPerPageRequested,
@QueryParam("start") final int paginationStart,
@QueryParam("show_relevance") boolean showRelevance,
@QueryParam("show_facets") boolean showFacets,
@QueryParam("fq") final List<String> filterQueries,
@QueryParam("show_entity_ids") boolean showEntityIds,
@QueryParam("show_api_urls") boolean showApiUrls,
@QueryParam("show_my_data") boolean showMyData,
@QueryParam("query_entities") boolean queryEntities,

I suggest to add a new parameter (name proposal: metadatafields ) to customize the returned items object.

The syntax of this new parameter could be something like :
metadataset_name:(*|metadata_field_name)[,metadataset_name:(*|metadata field name)]*

This way we could request all metadata fields from a metadataset using : geospatial:*
Or we could request a specific metadata field using : geospatial:bbox
And we could request metadata fields from different metadatasets using : geospatial:*,custom:data1

Impact of this new behaviour could be :

• edu.harvard.iq.dataverse.api.Search.search to add new parameter
• edu.harvard.iq.dataverse.search.SolrSearchResult.toJsonObject(boolean, boolean, boolean) to add the requested metadata fields parameter
• edu.harvard.iq.dataverse.search.SolrSearchResult.json(boolean, boolean, boolean) to add the requested metadata fields and to add fields to add to json Object.

What do you think about this ?

[djbrooke]

Hi @FNI18300 - do you have interest in making a Pull Request for this functionality? If you'd like, we're happy to discuss briefly before you get started. Or, if you feel it's a small change, go ahead and make the code change and we can discuss it in the Pull Request. Thanks!

[FNI18300]

Hi,
Yes I think I could create a Pull Request.
In fact, the code I quite ready but I'am a newbie with dataverse so I need some help to test/build and so on.

Since I use dataverse for my project, I use a docker embedded release from https://github.com/IQSS/dataverse-docker
The release of docker-image I use is 5.1.1-cv.
I'd like to upgrade this installation to 5.4.1 but deploying a war 5.4.1 seems to be insufficient.

Could you help me to start ? Do you have any tips ?

Thanks in advance

[djbrooke]

Hi @FNI18300, it's great to hear about the PR.

We maintain a list of instructions here: https://guides.dataverse.org/en/latest/developers/dev-environment.html

dataverse-docker is a separate project that's currently community-maintained, so I'd raise an issue there about the upgrade: https://github.com/IQSS/dataverse-docker

[pdurbin]

@FNI18300 like @djbrooke said, all the core developers use those instructions to get our dev environment set up. That is, we install Payara, Postgres, and Solr directly on our Macs.

That said, we do have a Docker image called "docker-aio" (for all-in-one) that you can try using if you want:

• https://guides.dataverse.org/en/5.4.1/developers/testing.html#running-the-full-api-test-suite-using-docker
• https://github.com/IQSS/dataverse/tree/develop/conf/docker-aio

[FNI18300]

HI,
and sorry this message, but I'm turning crazy
I try to deploy the latest release of dataverse into a docker with payara (payara/server-full:5.2020.7-jdk11) to test my code for the PR but the deployment failed with the following error :

Exception during lifecycle processing
dataverse2       | org.glassfish.deployment.common.DeploymentException: CDI definition failure:null -- null
dataverse2       |      at org.jboss.weld.bootstrap.events.ContainerLifecycleEvents.fireProcessAnnotatedType(ContainerLifecycleEvents.java:189)
dataverse2       |      at org.jboss.weld.bootstrap.events.ContainerLifecycleEvents.fireProcessAnnotatedType(ContainerLifecycleEvents.java:173)
dataverse2       |      at org.jboss.weld.bootstrap.BeanDeployer.processAnnotatedTypes(BeanDeployer.java:166)

As you can see, the message is very short....

I can see many EJB loaded during the deployement
I can see that the flyway job is done (the tables are created into database and the flyway history is correctly set)

I try to enable CDI Development Mode, I enabled and checked all the logs, But I could not find any tips to manage the exception.

How can I solve that ?

[pdurbin]

@FNI18300 sorry to hear you're having trouble. Sometimes these deployment woes drive me crazy too. 😄

Instead of troubleshooting here in this issue about search, let's move the conversation to a more general place, either a dedicated issue (please feel free to open one) or the "dev" mailing list at https://groups.google.com/g/dataverse-dev .

I did a quick search for the error but couldn't find a fix. I see you're using 5.2020.7 but I'm on 5.2020.6 and I'm not using Docker. Rather, I have everything installed on my Mac as described at https://guides.dataverse.org/en/5.5/developers/dev-environment.html . I don't know if it's possible for you to switch to a non-Docker environment or not.

We do have a Docker environment in the code base if you'd like to try it. It's called docker-aio (for all in one) and you can find it at https://github.com/IQSS/dataverse/tree/develop/conf/docker-aio

I hope this helps. Again, let's pick this up in a dedicated issue or thread. Please just leave a link here to that issue or thread once you've created it. Thanks.

[FNI18300]

Hi everybody,

I've you seen that I've created a pull request for my needs ? do you think we could merge it ?

[djbrooke]

Hi @FNI18300, thanks for the PR! We'll take a look soon. Apologies for the delay.