The SAML assertion for "Shib-Identity-Provider" was null.

[TanayKarve]

I am trying to integrate Shibboleth with dataverse.
I have the dataverse and shibboleth running as different services behind a reverse proxy.
I followed all steps in depth from the guide.
/Shibboleth.sso/Session has all the appropriate values.
I even looked into a similar issue on the google group, but my shibboleth2.xml does have AJP_ prefix set.
Even confirmed that attribute-map.xml is same as from the guide.

However, after logging in from my IDP, I am redirected to the blank account information page, and the logs say this:
The SAML assertion for "Shib-Identity-Provider" was null. Please contact support.

Am I missing something? How do I solve this?

[pdurbin]

@TanayKarve can you please post the output of /Shibboleth.sso/Session here? Or https://chat.dataverse.org

Or support@dataverse.org if you'd prefer to keep it private. Thanks!

[TanayKarve]

[pdurbin]

@TanayKarve thanks! Here's what I see:

Miscellaneous
Session Expiration (barring inactivity): 479 minute(s)
Client Address: 10.137.173.242
SSO Protocol: urn:oasis:names:tc:SAML:2.0:protocol
Identity Provider: https://fed.huit.harvard.edu/idp/shibboleth
Authentication Time: 2023-01-26T14:39:34.564Z
Authentication Context Class: urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport
Authentication Context Decl: (none)

Attributes
affiliation: staff@harvard.edu;member@harvard.edu;employee@harvard.edu
displayName: Philip Durbin
eppn: 7af9aa0d0cbfc97e@harvard.edu
givenName: Philip
mail: philip_durbin@harvard.edu
sn: Durbin

Yours also has "Identity Provider" so I'm not sure why it isn't working. (I'm assuming you change the real value to "mysite.edu" but that's fine.)

It would be nice to see what this method prints out:

    public static void printAttributes(HttpServletRequest request) {
        List<String> shibValues = new ArrayList<>();
        if (request == null) {
            logger.fine("HttpServletRequest was null. No shib values to print.");
            return;
        }
        for (String attr : shibAttrs) {

            /**
             * @todo explain in Installers Guide that in order for these
             * attributes to be found attributePrefix="AJP_" must be added to
             * /etc/shibboleth/shibboleth2.xml like this:
             *
             * <ApplicationDefaults entityID="https://dataverse.org/shibboleth"
             * REMOTE_USER="eppn" attributePrefix="AJP_">
             *
             */
            Object attrObject = request.getAttribute(attr);
            if (attrObject != null) {
                shibValues.add(attr + ": " + attrObject.toString());
            }
        }
        logger.fine("shib values: " + shibValues);
    }

Can you please increase the logging level like this...

./asadmin set-log-levels edu.harvard.iq.dataverse.authorization.providers.shib.ShibUtil=FINE

... as explained at https://guides.dataverse.org/en/5.12.1/admin/troubleshooting.html#increasing-payara-logging

Then, please try again and look for "shib values" in server.log.

[TanayKarve]

Hmm, strange. I see this for shib values:
shib values: []

and I do have this line in my /etc/shibboleth/shibboleth2.xml:
<ApplicationDefaults entityID="http://dataplanet.mysite.edu/shibboleth-sp" REMOTE_USER="eppn" attributePrefix="AJP_">

[pdurbin]

I know the comment says "change the entityID to your hostname"...

    <!-- FIXME: change the entityID to your hostname. -->
    <ApplicationDefaults entityID="https://dataverse.example.edu/sp"
        REMOTE_USER="eppn" attributePrefix="AJP_">

... but really it should be whatever your entityId is. (A pull request to fix this comment would be very welcome!) From your earlier screenshot it seems like you have the older style that doesn't look like a URL.

MIT's is urn:mace:incommon:mit.edu for example, see https://incommon.org/custom/federation/info/entity.html?entityID=urn%3Amace%3Aincommon%3Amit.edu&technical=true

Assuming yours is the older, more distinguished urn:mace... form, can you please try it, like this...

    <ApplicationDefaults entityID="urn:mace:incommon:mit.edu"
        REMOTE_USER="eppn" attributePrefix="AJP_">

[TanayKarve]

@pdurbin So I changed entityID to urn:mace:incommon:mysite.edu, but my IDP organisation said that entityID needs to be dataplanet.mysite.edu/shibboleth-sp because urn:mace:incommon:mysite.edu is the id of the IDP.
They confirmed that SAML response is correctly being sent with all the values.
Do you think it might be an issue with my reverse proxy? Here is my shib.conf:

# https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPApacheConfig

# RPM installations on platforms with a conf.d directory will
# result in this file being copied into that directory for you
# and preserved across upgrades.

# For non-RPM installs, you should copy the relevant contents of
# this file to a configuration location you control.
# don't pass paths used by Shibboleth to Payara
ProxyPassMatch ^/Shibboleth.sso !
ProxyPassMatch ^/shibboleth-ds !
# # pass everything else to Payara
#ProxyPass / ajp://localhost:8009/

#
# Load the Shibboleth module.
#
LoadModule mod_shib /usr/lib64/shibboleth/mod_shib_24.so

#
# Turn this on to support "require valid-user" rules from other
# mod_authn_* modules, and use "require shib-session" for anonymous
# session-based authorization in mod_shib.
#
ShibCompatValidUser On

#
# Ensures handler will be accessible.
#
<Location /Shibboleth.sso>
  AuthType None
  Require all granted
</Location>

#
# Used for example style sheet in error templates.
#
<IfModule mod_alias.c>
  <Location /shibboleth-sp>
    AuthType None
    Require all granted
  </Location>
  Alias /shibboleth-sp/main.css /usr/share/shibboleth/main.css
</IfModule>

#
# Configure the module for content.
#
# You MUST enable AuthType shibboleth for the module to process
# any requests, and there MUST be a require command as well. To
# enable Shibboleth but not specify any session/access requirements
# use "require shibboleth".
#
<Location /shib.xhtml>
  AuthType shibboleth
  ShibRequestSetting requireSession 1
  require valid-user
</Location>
# 

[pdurbin]

@TanayKarve woof. Honestly, as this point, I would try with https://samltest.id

It's been a little while since I tried but it should work. I described SAMLtest here: https://guides.dataverse.org/en/5.12.1/installation/shibboleth.html#exchange-metadata-with-your-identity-provider

The reason I added it to the guides is that sometimes people (like me) have trouble with their institutional identity provider and verifying that Dataverse works with SAMLtest helps people (like me) feel confident that they've configured Dataverse properly.

From a quick look your config looks fine but I may try to drag @donsizemore into this because he has much more current operational experience with Shib.

Anyway, please hang in there. We'll figure this out. At some point we can always reach out to the shib mailing list, which has been very helpful: https://shibboleth.net/mailman/listinfo/users

[donsizemore]

@TanayKarve Just curious, why aren't you passing traffic to Payara?

[TanayKarve]

@donsizemore okay, just uncommented #ProxyPass / ajp://localhost:8009/
but this is in shib.conf which is in a seperate container for shibboleth.
dataverse is in a seperate container which has payara running.
both services are under a reverse proxy using traefik
Edit:
So instead of localhost, I changed it to
ProxyPass / ajp://dataplanet.mysite.edu:8009

P.S. We're naming our deployment DataPlanet instead of Dataverse.

[pdurbin]

@TanayKarve since you're into modern things like containers, another thought for you: 😄

• Set up OIDC on the Dataverse side: https://guides.dataverse.org/en/5.12.1/installation/oidc.html
• Set up Keycloak. (See 9228 - add OIDC development setup for OIDC login feature testing #9234 but please note that it's just for dev and had a hard coded secret in it)
• Configure (in Dataverse) OIDC to talk to Keycloak.
• Configure Keycloak to talk to your institution's identity provider (using SAML)
• Profit!

No shibd anymore. Dataverse <-> OIDC <-> Keycloak <-> your IdP.

This is the direction we're going in the future anyway, probably. 😄 (See this auth doc).

@poikilotherm has something similar at https://data.fz-juelich.de . OIDC but Unity instead of Keycloak.

[TanayKarve]

How do I see if Payara is receiving data?
Also, in the logs I see this:
| JkEnvVar: null|#]

[donsizemore]

There's an HTTP header that's not getting passed between Traefik and Shibboleth, but I forget what it is.

@4tikhonov solved this exact problem and described it to Phil and to me in the Community Slack, but it's the free version and I can't read back that far. Let me Google a bit.

[pdurbin]

@4tikhonov solved this exact problem and described it to Phil and to me in the Community Slack, but it's the free version and I can't read back that far.

Gah! This is exactly why I opened this issue:

• dataversecommunity.slack.com has only 90 days of history #8712

As I said there, my vote is to shut down the community Slack and instead use https://chat.dataverse.org because it's SLOPI (and free, and based on standards, etc.). I give everyone a little Matrix/Element badge at https://people.dataverse.org if they join. 😄

The eventual goal is SMACKI, some sort of searchable knowledge base, but we should start with transparent communication.

Anyway, let me go search my email. And I'll bug Slava next week to see if he can remember. Thanks, Don!

[pdurbin]

Also, in the logs I see this:
| JkEnvVar: null|#]

At https://groups.google.com/g/dataverse-community/c/avpF_dPQOPo/m/OYpE0tSsAAAJ @mbamouni from https://entrepot.recherche.data.gouv.fr (which also has a fancy containerized setup that @edzale and @tjouneau described in a recent community meeting) wrote:

"I solved my problem using AJP to make reverse proxy in httpd"

Does that help?