`stat` is being launched for every file that is scanned.

[StyXman]

I just installed NC 13.0.0 and I noticed that it was taking a lot to scan my photos. I attached an strace the the scanner and I found out that's it calling stat for each file! I'm not talking about the C or PHP function, I'm talking about coreutils' /usr/bin/stat!

This has a huge performance impact in my server, and looking at the code, I can't figure out why this is tried before using the PHP function filesize(). And why it isn't even using [stat](https://secure.php.net/manual/en/function.stat.php)? Not to mention that is not calling escapeshellarg() either.

[StyXman]

Samething with getFileMtime().

[juliusknorr]

@StyXman Are you running NC on a 32-bit system? It seems that the exec is used there to workaround integer size issues. Anyway the arguments are properly escaped there using escapeshellarg

server/lib/private/LargeFileHelper.php

Line 148 in c1e4f9f

$arg = escapeshellarg($filename);

[marco44]

Disclaimer: I'm not a PHP developper. So I may miss the point.

But since I faced this issue, or something similar, I gave it a look...
If I read correctly, to get filesize, the code should use curl before doing an exec for file size, which should avoid an exec ? It doesn't in my environment, so I feel something is wrong with getFileSizeViaCurl (curl is activated in php.ini)

I still have a lot of execs. Because this is also used for getting mtime, using stat %Y. And this seems to be a waste of resources, as I should be ok until 2038 on this small arm device :)

Anyway, this has a huge cost in my use case. Commenting both execs made the load go from 4 CPUs to 10% of one in this environment...

[marco44]

Found out why the getFileSizeViaCurl doesn't work (at least on my environment).

I get this error * Invalid file://hostname/, expected localhost or 127.0.0.1 or none

It seems that parsing from cURL has changed a little bit in recent versions (I saw some reports on this for cURL 7.59 I'm using). It seems to be to conform to https://www.ietf.org/rfc/rfc1738.txt , which states that a file:// URL must contain a host, between the second and third /. So if you don't have a third / at all, it errors out. The way to avoid that is to put three slashes straight after file:. Which I did… and it solved the problem, now file size is resolved using curl, which avoid two forks and two execs.

I also deactivated the exec in getFileMtime, I just don't understand what the point is to call stat to get a mtime, which should be valid for 20 more years without overflow, but as said before, I could have missed something, I'm no PHP developper.

[marco44]

Hi,

Here is a proposal... I can do a pull request if you tell me it seems ok...

diff_LargeFileHelper.txt

[marco44]

BTW the thing I missed: if you perform a getfilemtime on a file bigger than 2GB, PHP throws an exception… hence the exception block around it.

[juliusknorr]

@marco44 The changes make sense to me. It would be great if you could open a pull request for this so we can discuss the changes properly. 👍