Skip to content

WebUI: Serialize upload of files to prevent upload failures #18537

@csware

Description

@csware

Uploading several bigger files fails using the WebUI.

Steps to reproduce

  1. Login to Nextcloud in browser (such as Firefox)
  2. Select several bigger files locally and drag them into Nextcloud WebUI to upload

Expected behaviour

The files are all uploaded (sequentially) and the upload succeeds.

Actual behaviour

All files are uploaded in parallel. Only a part of the files are uploaded correctly. This is especially an issue with less bandwidth and gets even worse if Slowloris mitigations are enabled.

Server configuration

Operating system: Debian 9.11

Web server: Apache/2.4.25 (Debian)

Database: MariaDB 10.1.41-MariaDB

PHP version: 7.3.13

Nextcloud version: 17.0.2

Updated from an older Nextcloud/ownCloud or fresh install: updated

Where did you install Nextcloud from: Nextcloud.org server package

Signing status:

Signing status
No errors have been found.

Are you using external storage, if yes which one: local

Are you using encryption: no

Client configuration

Browser: Mozilla Firefox 71.0

Operating system: Win 10 x64

Logs

Web server error log

Web server error log
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-890c1b7aa7c4ffa0a62bc2ab7efe9443-1577023169303 HTTP/2.0" 201 367 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-3b5fb1e494cbb5e924b3ffa7053f0a85-1577023169303 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-bb803fbbee1f027a43c397be4e2de1c5-1577023169304 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-8350caf63503c4ecc78664f68ca604a2-1577023169304 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-4b3af404f126dbd45b982f0c1ff9fad3-1577023169303 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-eba065a5c899f4d6c34478a7727774b6-1577023169304 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-bea842cc9f4ffda3fd6c6480718ff639-1577023169305 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-83665f71f5a86d31116d5276eb9be313-1577023169305 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:32 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-e94160be7d3242f180bff890005a0485-1577023169305 HTTP/2.0" 201 61 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:32 +0100] "MKCOL /remote.php/dav/uploads/USERNAME/web-file-upload-5ab4530899554ffb48920ba7f3760f7e-1577023169305 HTTP/2.0" 201 38 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:32 +0100] "PUT /remote.php/dav/uploads/USERNAME/web-file-upload-e94160be7d3242f180bff890005a0485-1577023169305/0 HTTP/2.0" 201 4041 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
IP - - [22/Dec/2019:14:59:31 +0100] "PUT /remote.php/dav/uploads/USERNAME/web-file-upload-3b5fb1e494cbb5e924b3ffa7053f0a85-1577023169303/0 HTTP/2.0" 201 3942 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"
...

Nextcloud log (data/nextcloud.log)

Nextcloud log
{"reqId":"Kuve8QPS1lGKODpo76hy","level":4,"time":"2019-12-22T15:09:31+01:00","remoteAddr":"IP","user":"USERNAME","app":"webdav","method":"PUT","url":"\/remote.php\/dav\/uploads\/USERNAME\/web-file-upload-4b3af404f126dbd45b982f0c1ff9fad3-1577023169303\/62914560","message":{"Exception":"Sabre\\DAV\\Exception\\BadRequest","Message":"Expected filesize of 10485760 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 8642560 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.","Code":0,"Trace":[{"file":"\/var\/www\/PATH\/apps\/dav\/lib\/Connector\/Sabre\/Directory.php","line":156,"function":"put","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":[null]},{"file":"\/var\/www\/PATH\/apps\/dav\/lib\/Upload\/UploadFolder.php","line":45,"function":"createFile","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->","args":["62914560",null]},{"file":"\/var\/www\/PATH\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":1096,"function":"createFile","class":"OCA\\DAV\\Upload\\UploadFolder","type":"->","args":["62914560",null]},{"file":"\/var\/www\/PATH\/3rdparty\/sabre\/dav\/lib\/DAV\/CorePlugin.php","line":525,"function":"createFile","class":"Sabre\\DAV\\Server","type":"->","args":["uploads\/USERNAME\/web-file-upload-4b3af404f126dbd45b982f0c1ff9fad3-1577023169303\/62914560",null,null]},{"function":"httpPut","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[{"absoluteUrl":"https:\/\/PATH\/remote.php\/dav\/uploads\/USERNAME\/web-file-upload-4b3af404f126dbd45b982f0c1ff9fad3-1577023169303\/62914560","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"\/var\/www\/PATH\/3rdparty\/sabre\/event\/lib\/EventEmitterTrait.php","line":105,"function":"call_user_func_array","args":[[{"__class__":"Sabre\\DAV\\CorePlugin"},"httpPut"],[{"absoluteUrl":"https:\/\/PATH\/remote.php\/dav\/uploads\/USERNAME\/web-file-upload-4b3af404f126dbd45b982f0c1ff9fad3-1577023169303\/62914560","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"\/var\/www\/PATH\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":479,"function":"emit","class":"Sabre\\Event\\EventEmitter","type":"->","args":["method:PUT",[{"absoluteUrl":"https:\/\/PATH\/remote.php\/dav\/uploads\/USERNAME\/web-file-upload-4b3af404f126dbd45b982f0c1ff9fad3-1577023169303\/62914560","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"\/var\/www\/PATH\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":254,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"absoluteUrl":"https:\/\/PATH\/remote.php\/dav\/uploads\/USERNAME\/web-file-upload-4b3af404f126dbd45b982f0c1ff9fad3-1577023169303\/62914560","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"\/var\/www\/PATH\/apps\/dav\/lib\/Server.php","line":317,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"\/var\/www\/PATH\/apps\/dav\/appinfo\/v2\/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"\/var\/www\/PATH\/remote.php","line":163,"args":["\/var\/www\/PATH\/apps\/dav\/appinfo\/v2\/remote.php"],"function":"require_once"}],"File":"\/var\/www\/PATH\/apps\/dav\/lib\/Connector\/Sabre\/File.php","Line":228,"CustomMessage":"--"},"userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko\/20100101 Firefox\/71.0","version":"17.0.2.1"}

Browser log

Browser log

I see the following warnings approx. 29K times:

The humanFileSize library is deprecated! It will be removed in nextcloud 19. 3 main.js:1095:57
The escapeHTML library is deprecated! It will be removed in nextcloud 19. 3 main.js:1095:57

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions