External Sites App: New user(s) cannot access a defined external app.

[kneiser]

Steps to reproduce

1. External Sites application installed and related to a security group:

Site Name: Scanner
URL: https://site_IP_address/sites
Language: English (US)
Groups: scanGrp
Devices: Only in the browser
Icon: scanner.svg
Position: Header
Redirect: not checked

2. Administrator created new user and assigned "scanGrp" membership.
3. New user logs in with no problem.

Expected behaviour

After a user logs in (scanGrp assigned) the Scanner icon should be visible in the NC application bar.

Actual behaviour

1. New user logs in no Scanner icon is visible in NC application bar.
2. Existing users, prior to new user creation, have scanner icon and can access the external site as defined.
3. When creating a "test user" on same problem occurs.

Suspect no problem with the External Sites setup, must be a data problem in MariaDB table(s)?

Server configuration detail

Operating system: Linux 4.9.0-7-686-pae #1 SMP Debian 4.9.110-1 (2018-07-05) i686

Webserver: Apache/2.4.25 (Debian) (apache2handler)

Database: mysql 10.1.26 (MariaDB)

PHP version: 7.0.30-0+deb9u1
Modules loaded: Core, date, libxml, openssl, pcre, zlib, filter, hash, Reflection, SPL, session, standard, apache2handler, imagick, mysqlnd, PDO, xml, apcu, apc, bz2, calendar, ctype, curl, dom, mbstring, fileinfo, ftp, gd, gettext, iconv, intl, json, exif, mcrypt, mysqli, pdo_mysql, Phar, posix, readline, shmop, SimpleXML, sockets, sysvmsg, sysvsem, sysvshm, tokenizer, wddx, xmlreader, xmlwriter, xsl, zip, Zend OPcache

Nextcloud version: 13.0.5 - 13.0.5.2

Updated from an older Nextcloud/ownCloud or fresh install: Nextcloud

Where did you install Nextcloud from: Via NC updater.

Signing status

Array

List of activated apps

Enabled:
 - apporder: 0.4.1
 - audioplayer: 2.3.1
 - bookmarks: 0.11.0
 - bruteforcesettings: 1.1.0
 - calendar: 1.6.1
 - caniupdate: 0.1.3
 - comments: 1.3.0
 - contacts: 2.1.5
 - dav: 1.4.7
 - deck: 0.4.1
 - drawio: 0.9.0
 - external: 3.0.3
 - federatedfilesharing: 1.3.1
 - federation: 1.3.0
 - files: 1.8.0
 - files_external: 1.4.1
 - files_markdown: 2.0.4
 - files_pdfviewer: 1.2.1
 - files_reader: 1.2.3
 - files_sharing: 1.5.0
 - files_texteditor: 2.5.1
 - files_trashbin: 1.3.0
 - files_videoplayer: 1.2.0
 - flowupload: 0.0.7
 - gallery: 18.0.0
 - groupfolders: 1.2.2
 - issuetemplate: 0.3.0
 - logreader: 2.0.0
 - lookup_server_connector: 1.1.0
 - metadata: 0.6.0
 - nextcloud_announcements: 1.2.0
 - notifications: 2.1.2
 - oauth2: 1.1.1
 - password_policy: 1.3.0
 - provisioning_api: 1.3.0
 - serverinfo: 1.3.0
 - sharebymail: 1.3.0
 - survey_client: 1.1.0
 - systemtags: 1.3.0
 - theming: 1.4.5
 - twofactor_backupcodes: 1.2.3
 - updatenotification: 1.3.0
 - weather: 1.5.1
 - workflowengine: 1.3.0
Disabled:
 - activity
 - admin_audit
 - encryption
 - files_versions
 - firstrunwizard
 - user_external
 - user_ldap

Configuration (config/config.php)

{
    "instanceid": "***REMOVED SENSITIVE VALUE***",
    "passwordsalt": "***REMOVED SENSITIVE VALUE***",
    "secret": "***REMOVED SENSITIVE VALUE***",
    "trusted_domains": [
        "REMOVED SENSITIVE VALUE"
    ],
    "datadirectory": "***REMOVED SENSITIVE VALUE***",
    "overwrite.cli.url": "http:\/\/REMOVED SENSITIVE VALUE\/cloud",
    "overwriteprotocol": "https",
    "dbtype": "mysql",
    "version": "13.0.5.2",
    "dbname": "***REMOVED SENSITIVE VALUE***",
    "dbhost": "***REMOVED SENSITIVE VALUE***",
    "dbport": "",
    "dbtableprefix": "oc_",
    "log_rotate_size": "100 MiB",
    "memcache.local": "\\OC\\Memcache\\APCu",
    "trashbin_retention_obligation": "auto, 7",
    "defaultapp": "apporder",
    "dbuser": "***REMOVED SENSITIVE VALUE***",
    "dbpassword": "***REMOVED SENSITIVE VALUE***",
    "installed": true,
    "maintenance": false,
    "theme": "",
    "loglevel": 2,
    "updater.secret": "***REMOVED SENSITIVE VALUE***"
}

Are you using external storage, if yes which one: local

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

LDAP configuration (delete this part if not used)

With access to your command line run e.g.:
sudo -u www-data php occ ldap:show-config
from within your Nextcloud installation folder

Without access to your command line download the data/owncloud.db to your local
computer or access your SQL server remotely and run the select query:
SELECT * FROM `oc_appconfig` WHERE `appid` = 'user_ldap';


Eventually replace sensitive data as the name/IP-address of your LDAP server or groups.

Client configuration

Browser: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0

Operating system: Debian 9.5

Logs

Browser log

{"reqId":"OpTRYlQ7kBMKcQG4FL1C","level":2,"time":"2018-08-01T03:42:55+00:00","remoteAddr":"192.168.0.151","user":"--","app":"OC\\Log\\Rotate","method":"GET","url":"\/cloud\/cron.php","message":"Log file \"\/var\/files\/cloud\/data\/nextcloud.log\" was over 100 MiB bytes, moved to \"\/var\/files\/cloud\/data\/nextcloud.log.1\"","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko\/20100101 Firefox\/61.0","version":"13.0.5.2"}
{"reqId":"MO8S0f0GtaOB9twlf9JO","level":2,"time":"2018-08-01T03:43:09+00:00","remoteAddr":"192.168.0.151","user":"new_user","app":"l10n","method":"GET","url":"\/cloud\/index.php\/apps\/drawio\/ajax\/settings","message":"Failed to load \/var\/www\/cloud\/apps\/drawio\/l10n\/en_GB.json - json error code: 4","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko\/20100101 Firefox\/61.0","version":"13.0.5.2"}
{"reqId":"2pRpcNdb7kRatBIZPXcO","level":3,"time":"2018-08-01T03:43:30+00:00","remoteAddr":"192.168.0.151","user":"--","app":"PHP","method":"GET","url":"\/cloud\/index.php\/logout?requesttoken=fupqRfQytYq87s4NhdegpN1A\/avLsMVk6vHmRwhytgw%3D%3AOoUZJ6B2%2BMKIrLdX5pPM1JwHt%2BGd\/agIv7agbFIZ3ks%3D","message":"Module 'imagick' already loaded at Unknown#0","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko\/20100101 Firefox\/61.0","version":"13.0.5.2"}

Nextcloud log

{"reqId":"OpTRYlQ7kBMKcQG4FL1C","level":2,"time":"2018-08-01T03:42:55+00:00","remoteAddr":"192.168.0.151","user":"--","app":"OC\\Log\\Rotate","method":"GET","url":"\/cloud\/cron.php","message":"Log file \"\/var\/files\/cloud\/data\/nextcloud.log\" was over 100 MiB bytes, moved to \"\/var\/files\/cloud\/data\/nextcloud.log.1\"","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko\/20100101 Firefox\/61.0","version":"13.0.5.2"}
{"reqId":"MO8S0f0GtaOB9twlf9JO","level":2,"time":"2018-08-01T03:43:09+00:00","remoteAddr":"192.168.0.151","user":"new_user","app":"l10n","method":"GET","url":"\/cloud\/index.php\/apps\/drawio\/ajax\/settings","message":"Failed to load \/var\/www\/cloud\/apps\/drawio\/l10n\/en_GB.json - json error code: 4","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko\/20100101 Firefox\/61.0","version":"13.0.5.2"}
{"reqId":"2pRpcNdb7kRatBIZPXcO","level":3,"time":"2018-08-01T03:43:30+00:00","remoteAddr":"192.168.0.151","user":"--","app":"PHP","method":"GET","url":"\/cloud\/index.php\/logout?requesttoken=fupqRfQytYq87s4NhdegpN1A\/avLsMVk6vHmRwhytgw%3D%3AOoUZJ6B2%2BMKIrLdX5pPM1JwHt%2BGd\/agIv7agbFIZ3ks%3D","message":"Module 'imagick' already loaded at Unknown#0","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko\/20100101 Firefox\/61.0","version":"13.0.5.2"}
{"reqId":"5i5TzDSlUxV2ZgwQPgwD","level":3,"time":"2018-08-01T03:45:50+00:00","remoteAddr":"192.168.0.151","user":"admin","app":"no app in context","method":"GET","url":"\/cloud\/index.php\/settings\/user","message":"issuetemplate","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko\/20100101 Firefox\/61.0","version":"13.0.5.2"}
{"reqId":"J8T4RntahKFaFgUtG68D","level":3,"time":"2018-08-01T03:45:54+00:00","remoteAddr":"192.168.0.151","user":"admin","app":"no app in context","method":"GET","url":"\/cloud\/index.php\/settings\/user\/additional","message":"issuetemplate","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko\/20100101 Firefox\/61.0","version":"13.0.5.2"}
{"reqId":"8yqwpY4bzLUl6IQACkxm","level":3,"time":"2018-08-01T03:46:02+00:00","remoteAddr":"192.168.0.151","user":"admin","app":"no app in context","method":"GET","url":"\/cloud\/index.php\/settings\/admin\/logging","message":"issuetemplate","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko\/20100101 Firefox\/61.0","version":"13.0.5.2"}
{"reqId":"RIx0GygroOjnUwWgz3LB","level":3,"time":"2018-08-01T03:46:05+00:00","remoteAddr":"192.168.0.151","user":"admin","app":"PHP","method":"GET","url":"\/cloud\/index.php\/apps\/apporder\/getOrder","message":"Module 'imagick' already loaded at Unknown#0","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko\/20100101 Firefox\/61.0","version":"13.0.5.2"}
{"reqId":"7AYSmTXhWj9tKEg4u5DQ","level":3,"time":"2018-08-01T03:46:05+00:00","remoteAddr":"192.168.0.151","user":"admin","app":"PHP","method":"GET","url":"\/cloud\/ocs\/v2.php\/apps\/notifications\/api\/v2\/notifications","message":"Module 'imagick' already loaded at Unknown#0","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko\/20100101 Firefox\/61.0","version":"13.0.5.2"}

Browser log

Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log
c) ...

[nickvergessen]

Just tested and works fine here.

[kneiser]

@nickvergessen

I have a test system and it has the same problem. Both instances of the operating system and Nextcloud are configured the same. I see you added a "needs info" tag, what do you need?

[kneiser]

@nickvergessen
Would you need the information below by any chance?
Browser Console (F12-Console) has the following:

Content Security Policy: Directive ‘child-src’ has been deprecated. Please use directive ‘worker-src’ to control workers, or directive ‘frame-src’ to control frames respectively.
Content Security Policy: The page’s settings blocked the loading of a resource at self (“script-src”). Source: (function (NAVIGATOR, OBJECT) {

OBJ.... files:1

Content Security Policy: The page’s settings blocked the loading of a resource at self (“script-src”). Source: (function (DOCUMENT, dispatchEvent, CUST.... files:1
JQMIGRATE: Migrate is installed, version 1.4.0 core.js:7:542
window.controllers/Controllers is deprecated. Do not use it for UA detection. merged.js:2171
Source map error: TypeError: NetworkError when attempting to fetch resource.
Resource URL: https://123.123.123.123/cloud/core/vendor/core.js?v=50988cb6-2

Note: The icon for the external site does not appear for the new user.

[kneiser]

@nickvergessen
Since both of my test and production systems are having the same issue with External Sites, I decided to do some additional testing on my test system:

1. I removed all external sites defined.
2. I removed the External sites application through the Admin user.
3. I removed the group "scanGrp" from all user assignments.
4. I removed the additional new user but left all the existing users defined before the new user was created.
5. I rebooted the test server.
6. I installed the External Sites application.
7. I defined a simple external site (i.e. BBCNews) but did not assign a security group.
8. Tested all existing users, the new external site appeared in the Nextcloud application menu.
9. I created a new user, named differently than the other new user I removed.
10. The new user did not have the external site in the Nextcloud menu.

These additinal tests lead me to believe that perhaps I have some sort of data issues in one or more Nextcloud SQL database tables? Where should I look next as I am not familiar where external sites are stored in the database? Perhaps a comparision of an existing users' External Sites settings are in order? Please let me know as I would like to get this issue resolved as soon as possible.

[nickvergessen]

The sites are stored in the oc_appconfig table. There is only one set stored for all users.

[kneiser]

@nickvergessen

Is there a particular record type I should be looking for? If there is only one set of records stored for all users, why would newly created user IDs not have the external sites in their menu?

[kneiser]

@nickvergessen
In your last post you referred to one set of records in the database table "oc_appconfig". I did some poking around and here is what I found so far:

1. Please see attached screenshot of the record set.

2. Here is the detail for the "sites" record:

{"7":{"id":7,"name":"Weather-Stony Plain","url":"https:\/\/weather.gc.ca\/city\/pages\/ab-22_metric_e.html","lang":"en","type":"link","device":"browser","icon":"external.svg","groups":[],"redirect":false}}

3. Here are the user settings. The "test0" user has the external site icon in the menu and functions correctly. The "test4" user is the new user defined and does not have the external site icon. No security group has been associated with the external sites definition.

{"displayname":{"value":"test0","scope":"contacts","verified":"0"},"address":{"value":"","scope":"private","verified":"0"},"website":{"value":"","scope":"private","verified":"0"},"email":{"value":null,"scope":"contacts","verified":"0"},"avatar":{"scope":"contacts"},"phone":{"value":"","scope":"private","verified":"0"},"twitter":{"value":"","scope":"private","verified":"0"}}

{"displayname":{"value":"test4","scope":"contacts","verified":"0"},"address":{"value":"","scope":"private","verified":"0"},"website":{"value":"","scope":"private","verified":"0"},"email":{"value":null,"scope":"contacts","verified":"0"},"avatar":{"scope":"contacts"},"phone":{"value":"","scope":"private","verified":"0"},"twitter":{"value":"","scope":"private","verified":"0"}}

4. I have not manually adjusted any of the above database records, they were all created by the Nextcloud instance.

Can you please guide me in next steps in to discovering the problem please? I have searched the NC Forum and have not found any similar issues posted there.

[nickvergessen]

I honestly have no idea, the external app always asks for the groups live, there is no caching involved on rendering the displayed pages either.

[kneiser]

@nickvergessen
This is a very weird problem. Both instances of NC were installed from scratch and all users were entered using the NC interface. I've never done any tinkering with the NC code or database records. I am not really interested in reinstalling stuff from the beginning, especially the production instance.

Would any of your other NC peers know where to look?

[kneiser]

@nickvergessen
After the new user logs in, is there a way to determine if an error occurs as the icon menu is built? Maybe the problem has nothing to do with the external sites app?.

[nickvergessen]

Well if something goes wrong it should be logged in the nextcloud.log file in your data folder.

[kneiser]

@nickvergessen
I just did added a security group on the Deck application. It shows up fine for the original test0 user and the new user test4. I guess this tests shows that overall NC security is functioning fine when restricting app access by group. I will try testing the external sites app again and send a copy of the nextcloud.log file after the test4 user logs in. Any additional suggestions before I run that test?

[kneiser]

@nickvergessen
I upgraded NC on my test server to V14.0.0. I don't know what you guys did but the external sites application now functions correctly! I will be upgrading my production server shortly, if all goes well I will be closing this issue.