[18.0.0 Beta1] [Update] InvalidFieldNameException in oc_flow_operations during update/repair

[MichaIng]

Steps to reproduce

1. Update from 17.0.1.1 to 18.0.0 Beta1 via web-based updater
2. Running the update/repair steps, from web UI, the ones after new files have been installed already.

Expected behaviour

• No errors

Actual behaviour

• 8 errors like this

[no app in context] Error: Doctrine\DBAL\Exception\InvalidFieldNameException: An exception occurred while executing 'SELECT DISTINCT `class`, `entity`, `events` FROM `oc_flow_operations` WHERE `events` <> ?' with params ["[]"]:

SQLSTATE[42S22]: Column not found: 1054 Unknown column 'entity' in 'field list' at <<closure>>

 0. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php line 169
    Doctrine\DBAL\Driver\AbstractMySQLDriver->convertException("An exception oc ... '", Doctrine\DBAL\Dr ... ]})
 1. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php line 145
    Doctrine\DBAL\DBALException::wrapException(Doctrine\DBAL\Driver\PDOMySql\Driver {}, Doctrine\DBAL\Dr ... ]}, "An exception oc ... '")
 2. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php line 911
    Doctrine\DBAL\DBALException::driverExceptionDuringQuery(Doctrine\DBAL\Driver\PDOMySql\Driver {}, Doctrine\DBAL\Dr ... ]}, "SELECT DISTINCT ... ?", {1: "[]"})
 3. /var/www/nextcloud/lib/private/DB/Connection.php line 196
    Doctrine\DBAL\Connection->executeQuery("SELECT DISTINCT ... ?", ["[]"], [2], null)
 4. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php line 200
    OC\DB\Connection->executeQuery("SELECT DISTINCT ... 1", {dcValue1: "[]"}, {dcValue1: 2})
 5. /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php line 214
    Doctrine\DBAL\Query\QueryBuilder->execute()
 6. /var/www/nextcloud/apps/workflowengine/lib/Manager.php line 137
    OC\DB\QueryBuilder\QueryBuilder->execute()
 7. /var/www/nextcloud/apps/workflowengine/lib/AppInfo/Application.php line 86
    OCA\WorkflowEngine\Manager->getAllConfiguredEvents()
 8. /var/www/nextcloud/apps/workflowengine/appinfo/app.php line 27
    OCA\WorkflowEngine\AppInfo\Application->registerRuleListeners()
 9. /var/www/nextcloud/lib/private/legacy/app.php line 264
    require_once("/var/www/nextcl ... p")
10. /var/www/nextcloud/lib/private/legacy/app.php line 155
    OC_App::requireAppFile("workflowengine")
11. /var/www/nextcloud/lib/private/legacy/app.php line 128
    OC_App::loadApp("workflowengine")
12. /var/www/nextcloud/lib/private/legacy/util.php line 196
    OC_App::loadApps(["filesystem"])
13. /var/www/nextcloud/lib/private/Files/Filesystem.php line 266
    OC_Util::setupFS("")
14. /var/www/nextcloud/lib/private/Server.php line 293
    OC\Files\Filesystem::getMountManager(null)
15. /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php line 118
    OC\Server->OC\{closure}("*** sensitive parameters replaced ***")
16. /var/www/nextcloud/lib/private/ServerContainer.php line 121
    Pimple\Container->offsetGet("RootFolder")
17. /var/www/nextcloud/lib/private/Server.php line 315
    OC\ServerContainer->query("RootFolder")
18. <<closure>>
    OC\Server->OC\{closure}("*** sensitive parameters replaced ***")
19. /var/www/nextcloud/lib/private/Files/Node/LazyRoot.php line 62
    call_user_func(Closure {})
20. /var/www/nextcloud/lib/private/Files/Node/LazyRoot.php line 79
    OC\Files\Node\LazyRoot->__call("listen", ["\\OC\\Files","postWrite",Closure {}])
21. /var/www/nextcloud/apps/previewgenerator/lib/AppInfo/Application.php line 50
    OC\Files\Node\LazyRoot->listen("\\OC\\Files", "postWrite", Closure {})
22. /var/www/nextcloud/apps/previewgenerator/lib/AppInfo/Application.php line 40
    OCA\PreviewGenerator\AppInfo\Application->connectWatcher(OC\AppFramework\ ... {})
23. /var/www/nextcloud/apps/previewgenerator/appinfo/app.php line 23
    OCA\PreviewGenerator\AppInfo\Application->__construct()
24. /var/www/nextcloud/lib/private/legacy/app.php line 264
    require_once("/var/www/nextcl ... p")
25. /var/www/nextcloud/lib/private/legacy/app.php line 155
    OC_App::requireAppFile("previewgenerator")
26. /var/www/nextcloud/lib/private/legacy/app.php line 128
    OC_App::loadApp("previewgenerator")
27. /var/www/nextcloud/lib/private/legacy/util.php line 196
    OC_App::loadApps(["filesystem"])
28. /var/www/nextcloud/lib/private/Files/Filesystem.php line 266
    OC_Util::setupFS("")
29. /var/www/nextcloud/lib/private/Server.php line 293
    OC\Files\Filesystem::getMountManager(null)
30. /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php line 118
    OC\Server->OC\{closure}("*** sensitive parameters replaced ***")
31. /var/www/nextcloud/lib/private/ServerContainer.php line 121
    Pimple\Container->offsetGet("RootFolder")
32. /var/www/nextcloud/lib/private/Server.php line 315
    OC\ServerContainer->query("RootFolder")
33. <<closure>>
    OC\Server->OC\{closure}("*** sensitive parameters replaced ***")
34. /var/www/nextcloud/lib/private/Files/Node/LazyRoot.php line 62
    call_user_func(Closure {})
35. /var/www/nextcloud/lib/private/Files/Node/LazyRoot.php line 142
    OC\Files\Node\LazyRoot->__call("get", ["appdata_ocv2j0skx6hk/js/core"])
36. /var/www/nextcloud/lib/private/Files/AppData/AppData.php line 140
    OC\Files\Node\LazyRoot->get("appdata_ocv2j0skx6hk/js/core")
37. /var/www/nextcloud/lib/private/Template/JSCombiner.php line 95
    OC\Files\AppData\AppData->getFolder("core")
38. /var/www/nextcloud/lib/private/Template/JSResourceLocator.php line 114
    OC\Template\JSCombiner->process("/var/www/nextcloud", "core/js/merged-template-prepend.json", "core")
39. /var/www/nextcloud/lib/private/Template/JSResourceLocator.php line 70
    OC\Template\JSResourceLocator->cacheAndAppendCombineJsonIfExist("/var/www/nextcloud", "core/js/merged-template-prepend.json")
40. /var/www/nextcloud/lib/private/Template/ResourceLocator.php line 78
    OC\Template\JSResourceLocator->doFind("js/merged-template-prepend")
41. /var/www/nextcloud/lib/private/TemplateLayout.php line 342
    OC\Template\ResourceLocator->find(["core/js/dist/m ... "])
42. /var/www/nextcloud/lib/private/TemplateLayout.php line 170
    OC\TemplateLayout::findJavascriptFiles(["core/js/dist/m ... "])
43. /var/www/nextcloud/lib/private/legacy/template.php line 183
    OC\TemplateLayout->__construct("error", "")
44. /var/www/nextcloud/lib/private/Template/Base.php line 131
    OC_Template->fetchPage()
45. /var/www/nextcloud/lib/private/legacy/template.php line 299
    OC\Template\Base->printPage()
46. /var/www/nextcloud/remote.php line 80
    OC_Template::printErrorPage("Service unavailable", "", 503)
47. /var/www/nextcloud/remote.php line 166
    handleException(RemoteException {})

PROPFIND /nextcloud/remote.php/dav/calendars/Micha/hamburg-holidays/
from 95.81.18.51 at 2019-12-10T10:26:14+01:00

• Now I see the PROPFIND request, the other similar log entries show them for other calendars and contacts and there is as well:
  GET /nextcloud/
  and
  GET /nextcloud/index.php/core/js/oc.js?v=a5601bfb
• Two other probably related error log entries are in between, see below.

But I cannot see further issues that seem to be related.

Server configuration

Operating system: Raspbian Bullseye

Web server: Apache/2.4.41

Database: MariaDB 10.3.20

PHP version: 7.3.12-1

Nextcloud version: 18.0.0 Beta1

Updated from an older Nextcloud/ownCloud or fresh install: 17.0.1.1

Where did you install Nextcloud from: download.nextcloud.com + web-based updater

Signing status:

Signing status

No errors have been found.

List of activated apps:

App list

Enabled:
  - activity: 2.11.0
  - apporder: 0.8.0
  - calendar: 2.0.0-beta3
  - cloud_federation_api: 1.1.0
  - contacts: 3.1.6
  - dav: 1.14.0
  - federatedfilesharing: 1.8.0
  - files: 1.13.1
  - files_rightclick: 0.15.1
  - files_trashbin: 1.8.0
  - files_versions: 1.11.0
  - impersonate: 1.4.0
  - logreader: 2.3.0
  - lookup_server_connector: 1.6.0
  - nextcloud_announcements: 1.7.0
  - notes: 3.0.3
  - notifications: 2.6.0
  - oauth2: 1.6.0
  - photos: 19.0.0
  - previewgenerator: 2.2.0
  - provisioning_api: 1.8.0
  - ransomware_protection: 1.5.1
  - settings: 1.0.0
  - survey_client: 1.6.0
  - tasks: 0.11.3
  - twofactor_backupcodes: 1.7.0
  - updatenotification: 1.8.0
  - workflowengine: 2.0.0
Disabled:
  - accessibility
  - admin_audit
  - comments
  - encryption
  - federation
  - files_external
  - files_pdfviewer
  - files_sharing
  - files_videoplayer
  - firstrunwizard
  - password_policy
  - privacy
  - recommendations
  - serverinfo
  - sharebymail
  - support
  - systemtags
  - text
  - theming
  - user_ldap
  - viewer

Nextcloud configuration:

Config report

{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "my.domain.org"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "18.0.0.3",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "filelocking.enabled": true,
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0
        },
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "loglevel": 1,
        "logtimezone": "Europe\/Berlin",
        "trashbin_retention_obligation": "disabled",
        "versions_retention_obligation": "disabled",
        "skeletondirectory": "",
        "defaultapp": "apporder",
        "maintenance": false,
        "overwrite.cli.url": "https:\/\/my.domain.org\/nextcloud",
        "htaccess.RewriteBase": "\/nextcloud",
        "mail_smtpmode": "smtp",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpsecure": "ssl",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "theme": "",
        "mysql.utf8mb4": true,
        "updater.release.channel": "beta",
        "tempdirectory": "\/mnt\/sda\/ncdata\/tmp",
        "preview_max_x": "2048",
        "preview_max_y": "2048",
        "jpeg_quality": "60",
        "app_install_overwrite": [
            "apporder",
            "calendar",
            "impersonate",
            "ransomware_protection",
            "contacts"
        ],
        "updater.secret": "***REMOVED SENSITIVE VALUE***"
    }
}

Are you using external storage, if yes which one: no

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Client configuration

Browser: Opera developer 67.0.3523.0

Operating system: Windows 10 Pro N 1909 build 18363.476

Logs

Web server error log

Web server error log

nothing related

Nextcloud log (data/nextcloud.log)

Nextcloud log

[PHP] Error: Undefined index: name at /var/www/nextcloud/core/templates/update.admin.php#24

GET /nextcloud/
from 95.81.18.51 at 2019-12-10T10:26:14+01:00
...
[jsresourceloader] Error: Could not find resource js/config.js to load

GET /nextcloud/
from 95.81.18.51 at 2019-12-10T10:26:14+01:00

Browser log

Browser log

nothing related

[haup]

I am experiencing exactly the same error. My logging gets polluted of this message

[stevleibelt]

Same for me right now, just tried to update to 18.0.1 and that is the result.

My template around that lines is looking like this:

 20         <div class="infogroup">
 21             <span><?php p($l->t('These incompatible apps will be disabled:')) ?></span>
 22             <ul class="content appList">
 23                 <?php foreach ($_['incompatibleAppsList'] as $appInfo) { ?>
 24                 <li><?php p($appInfo['name']) ?> (<?php p($appInfo['id']) ?>)</li>
 25                 <?php } ?>
 26             </ul>
 27         </div

[stevleibelt]

What I've tried so far:

login on the server and run "php occ upgrade"

[...] 
Checking for update of app spreed in appstore
Update app spreed from appstore
Repair error: Repair step 'OCA\Talk\Migration\FixNamespaceInDatabaseTables' is unknown
Repair error: Repair step 'OCA\Talk\Migration\CreateHelpCommand' is unknown
Repair error: Repair step 'OCA\Talk\Migration\ClearResourceAccessCache' is unknown
Checked for update of app "spreed" in appstore 
Repair error: Repair step 'OCA\Talk\Migration\FixNamespaceInDatabaseTables' is unknown
Repair error: Repair step 'OCA\Talk\Migration\CreateHelpCommand' is unknown
Repair error: Repair step 'OCA\Talk\Migration\ClearResourceAccessCache' is unknown
Repair error: Repair step 'OCA\Talk\Migration\CreateHelpCommand' is unknown
Starting code integrity check...
[...]

Things are looking good so far.

Following the list of my applications.

php public/occ app:list
Enabled:
  - accessibility: 1.4.0
  - admin_audit: 1.8.0
  - analytics: 2.0.0
  - audioplayer: 2.9.0
  - bruteforcesettings: 1.5.0
  - calendar: 2.0.1
  - cloud_federation_api: 1.1.0
  - contacts: 3.1.8
  - dav: 1.14.0
  - encryption: 2.6.0
  - federatedfilesharing: 1.8.0
  - federation: 1.8.0
  - files: 1.13.1
  - files_automatedtagging: 1.8.0
  - files_markdown: 2.2.0
  - files_mindmap: 0.0.17
  - files_pdfviewer: 1.7.0
  - files_rightclick: 0.15.2
  - files_sharing: 1.10.1
  - files_videoplayer: 1.7.0
  - logreader: 2.3.0
  - lookup_server_connector: 1.6.0
  - mail: 1.1.2
  - news: 14.1.3
  - nextbackup: 19.12.1
  - nextcloud_announcements: 1.7.0
  - notifications: 2.6.0
  - oauth2: 1.6.0
  - password_policy: 1.8.0
  - photos: 1.0.0
  - polls: 1.1.6
  - previewgenerator: 2.2.0
  - privacy: 1.2.0
  - provisioning_api: 1.8.0
  - recommendations: 0.6.0
  - serverinfo: 1.8.0
  - settings: 1.0.0
  - sharebymail: 1.8.0
  - spreed: 8.0.4
  - support: 1.1.0
  - survey_client: 1.6.0
  - systemtags: 1.8.0
  - text: 2.0.0
  - theming: 1.9.0
  - twofactor_backupcodes: 1.7.0
  - updatenotification: 1.8.0
  - viewer: 1.2.0
  - workflowengine: 2.0.0

[Midek]

After upgrading from Nextcloud 17 to Nextcloud 18.0.1, occ upgrade errored out:

└> sudo -u http php ./occ upgrade               
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
Set log level to debug
Updating database schema
Updated database
Updating <workflowengine> ...
Doctrine\DBAL\Exception\InvalidFieldNameException: An exception occurred while executing 'UPDATE "oc_flow_operations" SET "entity" = ? WHERE "entity" = ''' with params ["OCA\\WorkflowEngine\\Entity\\File"]:

SQLSTATE[42703]: Undefined column: 7 ERROR:  column "entity" does not exist
LINE 1: ...DATE "oc_flow_operations" SET "entity" = $1 WHERE "entity" =...
                                                             ^
Update failed
Maintenance mode is kept active
Reset log level

Indeed, there was no column entity in flow_operations after the upgrade, so I created it on my own and now it looks like its working fine.
alter table oc_flow_operations add column entity character varying(256) not null;

[AetherCollective]

Indeed, there was no column entity in flow_operations after the upgrade, so I created it on my own and now it looks like its working fine.
alter table oc_flow_operations add column entity character varying(256) not null;

@stevleibelt I tried the same and unfortunately I wasn't able to reproduce your success. Did you try anything else in your attempt to fix this?

ncadmin@cloud:~$ sudo -u www-data php /var/www/nextcloud/occ upgrade
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
Set log level to debug
Updating database schema
Updated database
Updating <workflowengine> ...
Doctrine\DBAL\Exception\InvalidFieldNameException: An exception occurred while executing 'UPDATE "oc_flow_operations" SET "entity" = ? WHERE "entity" = ''' with params ["OCA\\Workfl owEngine\\Entity\\File"]:

SQLSTATE[42703]: Undefined column: 7 ERROR:  column "entity" does not exist
LINE 1: ...DATE "oc_flow_operations" SET "entity" = $1 WHERE "entity" =...
                                                             ^
Update failed
Maintenance mode is kept active
Reset log level
ncadmin@cloud:~$ sudo -u postgres psql
psql (10.12 (Ubuntu 10.12-0ubuntu0.18.04.1))
Type "help" for help.

postgres=# select * from oc_flow_operations;
 id | class | name | checks | operation | entity
----+-------+------+--------+-----------+--------
(0 rows)

Edit: I realized my problem. I forgot to select the nextcloud_db database.
sudo -u postgres psql nextcloud_db

[solracsf]

@MichaIng duplicate of #18265? Feel free to close if yes, as the other topic seems more advanced in the resolution.

[MichaIng]

@acsfer
Many thanks for linking. It seems that the originating issue is at least similar: Flow related database tables seem to be not reliably created or created too late or not fully created. However, in my case it was only a onetime error message, no follow up issues, while the issue you linked contains a broken instance.

I had a very similar other issue, related to a different database table: #18325
As said in both cases the issue is a minor one, not leading to any followup errors until now. Probably the required tables were created a bid too late, so that they were accessed during Nextcloud update process while not yet being (fully) created or such. I'll link the issue over just in case those are related at some point.

[Nowaker]

I confirm the following helped:

alter table oc_flow_operations add column entity character varying(256) not null

I encountered the problem on Nextcloud 18.0.4.

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.