Upload of .htaccess file via WebDAV results in InsufficientStorage exception

Edit by @MorrisJobke: For steps see #2860 (comment)

--

Steps to reproduce

1. Attempt to upload a file named .htaccess via WebDAV
2. Will return a quota error since it cant parse files correctly in the Sabre CheckQuota plugin
3. Circumvent CheckQuota plugin to return true always results now in " Could not rename part file to final file"

Expected behaviour

dot files should be treated the exact same since it's a virtual filesystem not served directly by apache.

Actual behaviour

Various Erorrs

Server configuration

Operating system:
CentOS 7.3.1611 (x64)

Web server:
Server version: Apache/2.4.23 (cPanel)
Server built: Dec 13 2016 13:28:05
Server's Module Magic Number: 20120211:61
Server loaded: APR 1.5.2, APR-UTIL 1.5.2
Compiled using: APR 1.5.2, APR-UTIL 1.5.2
Architecture: 64-bit
Server MPM: prefork
threaded: no
forked: yes (variable process count)
Server compiled with....
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses disabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=256
-D HTTPD_ROOT="/etc/apache2"
-D SUEXEC_BIN="/usr/sbin/suexec"
-D DEFAULT_PIDLOG="/var/run/apache2/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"

Database:
mysqld Ver 10.1.20-MariaDB for Linux on x86_64 (MariaDB Server)

PHP version:
ea-php-cli Copyright 2016 cPanel, Inc.
PHP 7.0.14 (cli) (built: Dec 13 2016 13:45:15) ( NTS )
Copyright (c) 1997-2016 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2016 Zend Technologies
with the ionCube PHP Loader (enabled) + Intrusion Protection from ioncube24.com (unconfigured) v6.0.4, Copyright (c) 2002-2016, by ionCube Ltd.
Nextcloud version: (see Nextcloud admin page)

Updated from an older Nextcloud/ownCloud or fresh install:
Updated from 10.x to 11.0.0 Beta to fix ANOTHER issue with dot files.

Where did you install Nextcloud from:
tar file downloaded from nextcloud site

Signing status:

Signing status

Integrity checker has been disabled. Integrity cannot be verified.

List of activated apps:

App list

Unable to run. Script won't run from command line due to cPanel's implementation of PHP

The content of config/config.php:

Config report

$CONFIG = array (
  'instanceid' => '[REDACTED]',
  'passwordsalt' => '[REDACTED]',
  'secret' => '[REDACTED]',
  'trusted_domains' => 
  array (
    0 => '[REDACTED]',
  ),
  'datadirectory' => '[REDACTED]/data',
  'overwrite.cli.url' => '[REDACTED]',
  'dbtype' => 'mysql',
  'version' => '11.0.0.10',
  'dbname' => '[REDACTED]',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => '[REDACTED]',
  'dbpassword' => '[REDACTED]',
  'logtimezone' => 'UTC',
  'installed' => true,
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'tls',
  'mail_smtphost' => '[REDACTED]',
  'mail_smtpport' => '25',
  'mail_from_address' => 'nextcloud',
  'mail_domain' => '[REDACTED]',
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => '[REDACTED]',
    'port' => 0,
  ),
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 2,
  'updater.release.channel' => 'beta',
  'updater.secret' => '[REDACTED]',
);

Are you using external storage, if yes which one: No

Are you using encryption: No

Are you using an external user-backend, if yes which one: No

Client configuration

Browser: WebDav client

Operating system: Synology

Logs

Web server error log

Web server error log

``` [REDACTED] - - [26/Dec/2016:11:25:07 -0600] "PUT /remote.php/dav/files/[REDACTED]/htdocs/vendor/plugins/jqueryupload/server/php/files/.htaccess HTTP/1.1" 500 223 "-" "-" ```

Nextcloud log (data/nextcloud.log)

Nextcloud log

``` {"reqId":"WGFSpBwXDUaOa8OAMaKeLQAAAAU","remoteAddr":"[REDACTED]","app":"webdav","message":"renaming part file to final file failed","level":3,"time":"2016-12-26T17:25:57+00:00","method":"PUT","url":"\/remote.php\/dav\/files\/[REDACTED]\/htdocs\/vendor\/plugins\/jqueryupload\/server\/php\/files\/.htaccess","user":"james","version":"11.0.0.10"} {"reqId":"WGFSpBwXDUaOa8OAMaKeLQAAAAU","remoteAddr":"[REDACTED]","app":"webdav","message":"Exception: {\"Message\":\"HTTP\\\/1.1 500 Could not rename part file to final file\",\"Exception\":\"Sabre\\\\DAV\\\\Exception\",\"Code\":0,\"Trace\":\"#0 \\\[REDACTED]\\\/public_html\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/Directory.php(137): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\File->put(Resource id #405)\\n#1 \\\[REDACTED]\\\/public_html\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(1072): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\Directory->createFile('.htaccess', Resource id #405)\\n#2 \\\[REDACTED]\\/public_html\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(525): Sabre\\\\DAV\\\\Server->createFile('files\\\/james\\\/Doc...', Resource id #405, NULL)\\n#3 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpPut(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#4 \\\[REDACTED]\\\/public_html\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#5 \\\[REDACTED]\\\/public_html\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(479): Sabre\\\\Event\\\\EventEmitter->emit('method:PUT', Array)\\n#6 \\\[REDACTED]\\\/public_html\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#7 \\\[REDACTED]\\/public_html\\\/apps\\\/dav\\\/lib\\\/Server.php(227): Sabre\\\\DAV\\\\Server->exec()\\n#8 \\\[REDACTED]\\\/public_html\\\/apps\\\/dav\\\/appinfo\\\/v2\\\/remote.php(30): OCA\\\\DAV\\\\Server->exec()\\n#9 \\\[REDACTED]\\\/public_html\\\/remote.php(165): require_once('\\\/[REDACTED]...')\\n#10 {main}\",\"File\":\"\\\[REDACTED]\\\/public_html\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/File.php\",\"Line\":187,\"User\":\"[REDACTED]\"}","level":4,"time":"2016-12-26T17:25:57+00:00","method":"PUT","url":"\/remote.php\/dav\/files\[REDACTED]/htdocs\/vendor\/plugins\/jqueryupload\/server\/php\/files\/.htaccess","user":"[REDACTED]","version":"11.0.0.10"}

</details>

[MorrisJobke]

Works here fine:

curl -X PUT -u admin:admin -i -H "OC-Total-Length: 8" -H "Content-Type: text/plain" -d "sdaasdfa" http://localhost/server/remote.php/webdav/.my.txt
HTTP/1.1 201 Created
Date: Fri, 20 Jan 2017 01:36:19 GMT
Server: Apache/2.4.23 (Unix) PHP/5.6.27
X-Powered-By: PHP/5.6.27
Set-Cookie: ocz4fssjxo5e=d1fr91ppukqd1a7knmcci73g62; path=/server; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: oc_sessionPassphrase=eJd3mkWojrdlENuelsfQZGsDOxtepRjyGbYdeazfBqT20mFDPI6h%2B83Xn842f%2Fv2QYonTVcyJ%2FBF8EpJ4UdL3Nt7bRql0LJRqENASsfurdZyZ4EtADwmsdgo2x%2Bndep%2B; path=/server; httponly
Content-Security-Policy: default-src 'none';
Set-Cookie: nc_sameSiteCookielax=true; path=/server; httponly;expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=lax
Set-Cookie: nc_sameSiteCookiestrict=true; path=/server; httponly;expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=strict
Set-Cookie: ocz4fssjxo5e=3nq2nvr1suob2lfdlsooirajk7; path=/server; HttpOnly
Set-Cookie: cookie_test=test; expires=Fri, 20-Jan-2017 02:36:20 GMT; Max-Age=3600
OC-FileId: 00000028ocz4fssjxo5e
Content-Length: 0
ETag: "59105d8a466a3170604960d78fa33707"
OC-ETag: "59105d8a466a3170604960d78fa33707"
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Robots-Tag: none
X-Frame-Options: SAMEORIGIN
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Content-Type: text/html; charset=UTF-8

What webdav client do you use?

Synology WebDav CloudSync app. It's like it was treating it as a part file and not the entire thing, also trying to upload files ending in .part dont show up in the GUI and thus dont get downloaded by other clients.

[MorrisJobke]

Synology WebDav CloudSync app. It's like it was treating it as a part file and not the entire thing, also trying to upload files ending in .part dont show up in the GUI and thus dont get downloaded by other clients.

We have special rules: if the file ends on .part we thread them as chunks and don't expose them. So for now it's not possible to upload .part files to nextcloud, because there are part of our internal chunking mechanism.

[MorrisJobke]

Attempt to upload a file named .htaccess via WebDAV

And uploading .htaccess files is also not supported, because they would allow to tweak apache configs :(

[MorrisJobke]

dot files should be treated the exact same since it's a virtual filesystem not served directly by apache.

It's not about dot files. It was because it was an .htaccess file. Let me adjust the issue title.

[MorrisJobke]

File named .htaccess:

$ curl -X PUT -u admin:admin -H "OC-Total-Length: 8" -H "Content-Type: text/plain" -d "sdaasdfasdf1" http://localhost/server/remote.php/webdav/.htaccess 
<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\InsufficientStorage</s:exception>
  <s:message/>
</d:error>

Other dot file:

$ curl -X PUT -u admin:admin -H "OC-Total-Length: 8" -H "Content-Type: text/plain" -d "sdaasdfasdf1" http://localhost/server/remote.php/webdav/.htaccess2 -i
HTTP/1.1 204 No Content

[MorrisJobke]

@icewind1991 @rullzer Any idea why the quota wrapper behaves like that?

And uploading .htaccess files is also not supported, because they would allow to tweak apache configs :(

If the data directory is outside of DocumentRoot (which should be a good common practice) .htaccess file upload should be allowed. Also, should generate a useful error if not allowed, like a 405 Error instead of 500 Internal Server Error.

We have special rules: if the file ends on .part we thread them as chunks and don't expose them. So for now it's not possible to upload .part files to nextcloud, because there are part of our internal chunking mechanism.

There's no reason for this since there are HTTP headers that satisfy the ability to chunk properly are available. I know there are a few specifically for nextcloud, but there are standard ones like 'Range'.

[MorrisJobke]

There's no reason for this since there are HTTP headers that satisfy the ability to chunk properly are available. I know there are a few specifically for nextcloud, but there are standard ones like 'Range'.

It's for upload not for download. Chunking is sadly not specified for WebDAV so we needed to go to build our own.

It's for upload not for download. Chunking is sadly not specified for WebDAV so we needed to go to build our own.

You guys may want to reference https://blog.sphere.chronosempire.org.uk/2012/11/21/webdav-and-the-http-patch-nightmare for potential solution explanations. Since SabreDAV is used in this project, there's already a solution built into SabreDAV.

Here's the potential solutions

• If your WebDAV server is Apache (or lighttpd!): use PUT with Content-Range, and ignore what RFC 5789 says about this being forbidden.
• If your WebDAV server is a recent SabreDAV: use PATCH with a SabreDAV-specific Content-Type.
• If your WebDAV server is anything else (nginx, IIS, ...) you're probably out of luck.