Files created right before watching starts make watching go into a loop

[KidkArolis]

I've noticed this strange behaviour where watcher loops for 10 seconds due to this code:

if(data) {
  var ts = data[0] === data[1] ? data[0] + FS_ACCURENCY : data[0];
  if(ts > startTime)
    watcher.emit("change", data[1] + FS_ACCURENCY);
}

watchpack/lib/DirectoryWatcher.js

Lines 202 to 205 in 7efdd93

	if(data) {
	var ts = data[0] === data[1] ? data[0] + FS_ACCURENCY : data[0];
	if(ts > startTime)
	watcher.emit("change", data[1] + FS_ACCURENCY);

What I'm doing in my script is basically creating a file in a temporary directory /temp/entry.js, creating a webpack compiler and run compiler.watch(). Now for 10 seconds (which is the default value of FS_ACCURENCY) this is what's happening:

• compilation succeeds
• Watching.prototype._done gets called
• recalling Watching.prototype.watch
• which immediately triggers a change event for the /temp/entry.js file, because it's < 10 seconds old
• back to step 1

Any pointers on how to fix this would be helpful. For example, this could be fixed by modifying the above code to this, but I'd need to stare at this for longer to understand what this block of code is conceptually responsible for.

if(ts > startTime + FS_ACCURENCY) {

[sokra]

FS_ACCURENCY should automatically adjust to your file system accuracy. This may not happen fast enough if you have few files and the files are created unlucky on a timestamp modulo 10s.

The watching may loop in a unlucky case, but this should not result in a different compilation hash. I. e. the webpack-dev-server doesn't trigger a update if the hash is equal.

[KidkArolis]

Ah, but Im experiencing this issue 100% of the time, because I am
programatically creating the entry.js file and starting the watch process
on it right away. So for 10s it keeps rebuilding continuously.
On Fri, 6 May 2016 at 13:12, Tobias Koppers notifications@github.com
wrote:

FS_ACCURENCY should automatically adjust
https://github.com/webpack/watchpack/blob/master/lib/DirectoryWatcher.js#L230-L239
to your file system accuracy. This may not happen fast enough if you have
few files and the files are created unlucky on a timestamp modulo 10s.

The watching may loop in a unlucky case, but this should not result in a
different compilation hash. I. e. the webpack-dev-server doesn't trigger a
update if the hash is equal.

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#25 (comment)

[KidkArolis]

Would be nice to resolve this without having to fork webpack or delay
watching for 10s. I wonder if I could create a file with an older
timestamp..
On Fri, 6 May 2016 at 18:29, Karolis Narkevičius karolis.n@gmail.com
wrote:

Ah, but Im experiencing this issue 100% of the time, because I am
programatically creating the entry.js file and starting the watch process
on it right away. So for 10s it keeps rebuilding continuously.
On Fri, 6 May 2016 at 13:12, Tobias Koppers notifications@github.com
wrote:

FS_ACCURENCY should automatically adjust
https://github.com/webpack/watchpack/blob/master/lib/DirectoryWatcher.js#L230-L239
to your file system accuracy. This may not happen fast enough if you have
few files and the files are created unlucky on a timestamp modulo 10s.

The watching may loop in a unlucky case, but this should not result in a
different compilation hash. I. e. the webpack-dev-server doesn't trigger a
update if the hash is equal.

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#25 (comment)

[Jessidhia]

@sokra "FS_ACCURENCY" only adjusts itself when you get an actual onChange event. When you first fire compiler.watch(), and the compilation is fast enough (say, one file with no bundled dependencies), assuming you saved the file in the last few seconds, watchpack will keep emitting change events until the difference gets large enough.

When chokidar is using fswatch (OSX), FS_ACCURENCY eventually settles on 2000 if you trigger some real change events (not sure why, fsevent timestamps are multiples of 1000).

Maybe FS_ACCURENCY should get a first adjustment on the nextTick handler; at least that would reduce the worst case to 2s as opposed to 10s. If the build is really fast (say, ~400ms per build), you can easily get 20+ rebuilds at the start if you touched any file before starting the watch process.

It'd also be nice if it got renamed to FS_PRECISION, but that's going off-topic :)

[rochdev]

We are having the same problem. A workaround we found is to manually change the mtime of the file before it gets picked up by Webpack, but it feels like just adding a hack on top of a hack.

[hrimhari]

This problem has been haunting our team for a while. Files are being stored on doInitialScan() 10 seconds in the future and this generates multiple webpack-dev-middleware builds until the future becomes the past. It seems to me that watchpack should compare mtimes between nextTick()s instead of trying to guess if the mtime is in the future or in the past.

[sokra]

Watchpacks strategy is: if not sure if it's a change, thread it as change. With low fs accuracy files could have changed even if mime is equal.

Normally multiple rebuilds shouldn't cause issues and this cause should be cached by the caller by comparing hash. I.e. the webpack CLI doesn't display stats output for equal hash.

Nevertheless this may could be fixed in watch pack by storing the time of the initial scan/watcher installation and useing this time.

[hrimhari]

How is FS_ACCURENCY helping in that case? If the filesystem doesn't provide the precision necessary to detect the change, FS_ACCURENCY will only add X milliseconds to that lack of precision. If mtime didn't change because of lack of precision, adding FS_ACCURENCY to it won't make it any different. The only reason I can imagine is when trying to guess if the mtime is in the future or not, and what I propose is to stop doing that and just act upon the change in mtime itself.

[hrimhari]

FYI, here's an extract of what we see if we have a recent change before starting webpack-dev-middleware. I put a loop printing the current date in the background and added some console.logs for debugging purposes showing arguments for the compiler plugins. Note that the hash remains the same and the multiple builds happening until startTime is greater than mtime + FS_ACCURENCY:

Tue 18 Oct 2016 11:33:20 EDT
webpack built a6a4935a84d4fc7fddd3 in 3823ms
Hash: a6a4935a84d4fc7fddd3
Version: webpack 2.1.0-beta.22
Time: 3823ms
                               Asset     Size  Chunks             Chunk Names
13ed6d2a0f1addfda9c8530374554c10.png    12 kB          [emitted]
007abab9d8f8ae99042650e9311bdc82.png  16.1 kB          [emitted]
7f86a225ce5b3f8879ef567e8ef37743.png  14.2 kB          [emitted]
           0.a6a4935a84d4fc7fddd3.js  14.5 kB       0  [emitted]
           1.a6a4935a84d4fc7fddd3.js  23.1 kB       1  [emitted]
         app.a6a4935a84d4fc7fddd3.js  1.46 MB       2  [emitted]  app
                         favicon.ico  7.41 kB          [emitted]
                       vendor.dll.js  3.92 MB          [emitted]
                          index.html  6.61 kB          [emitted]
Child html-webpack-plugin for "index.html":
                                   Asset     Size  Chunks             Chunk Names
    13ed6d2a0f1addfda9c8530374554c10.png    12 kB          [emitted]
                              index.html  1.46 MB       0
webpack: bundle is now VALID.
Tue 18 Oct 2016 11:33:21 EDT
invalid
{ '0': '/Users/fcarasso/Dev/console-proto/src/main.js',
  '1': 1476804803000 }
webpack: bundle is now INVALID.
invalidAsync
{ '0':
   Watching {
     startTime: 1476804802113,
     invalid: false,
     error: null,
     stats:
      Stats {
        compilation: [Object],
        hash: 'a6a4935a84d4fc7fddd3',
        startTime: 1476804797495,
        endTime: 1476804801318 },
     handler: [Function],
     watchOptions: { aggregateTimeout: 200 },
     compiler:
      Compiler {
        _plugins: [Object],
        outputPath: '/Users/fcarasso/Dev/console-proto/dist',
        outputFileSystem: [Object],
        inputFileSystem: [Object],
        recordsInputPath: undefined,
        recordsOutputPath: undefined,
        records: [Object],
        fileTimestamps: [Object],
        contextTimestamps: [Object],
        resolvers: [Object],
        parser: [Object],
        options: [Object],
        context: '/Users/fcarasso/Dev/console-proto',
        name: 'client',
        watchFileSystem: [Object],
        _lastCompilationFileDependencies: [Object],
        _lastCompilationContextDependencies: [] },
     running: true,
     watcher: null },
  '1': [Function: next] }
invalid
{}
webpack building...
webpack built a6a4935a84d4fc7fddd3 in 227ms
Hash: a6a4935a84d4fc7fddd3
Version: webpack 2.1.0-beta.22
Time: 227ms
                               Asset     Size  Chunks       Chunk Names
13ed6d2a0f1addfda9c8530374554c10.png    12 kB
007abab9d8f8ae99042650e9311bdc82.png  16.1 kB
7f86a225ce5b3f8879ef567e8ef37743.png  14.2 kB
           0.a6a4935a84d4fc7fddd3.js  14.5 kB       0
           1.a6a4935a84d4fc7fddd3.js  23.1 kB       1
         app.a6a4935a84d4fc7fddd3.js  1.46 MB       2       app
Child html-webpack-plugin for "index.html":
                                   Asset     Size  Chunks       Chunk Names
    13ed6d2a0f1addfda9c8530374554c10.png    12 kB
                              index.html  1.46 MB       0
webpack: bundle is now VALID.
invalid
{ '0': '/Users/fcarasso/Dev/console-proto/src/main.js',
  '1': 1476804803000 }
webpack: bundle is now INVALID.
invalidAsync
{ '0':
   Watching {
     startTime: 1476804802571,
     invalid: false,
     error: null,
     stats:
      Stats {
        compilation: [Object],
        hash: 'a6a4935a84d4fc7fddd3',
        startTime: 1476804802113,
        endTime: 1476804802340 },
     handler: [Function],
     watchOptions: { aggregateTimeout: 200 },
     compiler:
      Compiler {
        _plugins: [Object],
        outputPath: '/Users/fcarasso/Dev/console-proto/dist',
        outputFileSystem: [Object],
        inputFileSystem: [Object],
        recordsInputPath: undefined,
        recordsOutputPath: undefined,
        records: [Object],
        fileTimestamps: [Object],
        contextTimestamps: [Object],
        resolvers: [Object],
        parser: [Object],
        options: [Object],
        context: '/Users/fcarasso/Dev/console-proto',
        name: 'client',
        watchFileSystem: [Object],
        _lastCompilationFileDependencies: [Object],
        _lastCompilationContextDependencies: [] },
     running: true,
     watcher: null },
  '1': [Function: next] }
invalid
{}
webpack building...
Tue 18 Oct 2016 11:33:22 EDT
webpack built a6a4935a84d4fc7fddd3 in 220ms
Hash: a6a4935a84d4fc7fddd3
Version: webpack 2.1.0-beta.22
Time: 220ms
                               Asset     Size  Chunks       Chunk Names
13ed6d2a0f1addfda9c8530374554c10.png    12 kB
007abab9d8f8ae99042650e9311bdc82.png  16.1 kB
7f86a225ce5b3f8879ef567e8ef37743.png  14.2 kB
           0.a6a4935a84d4fc7fddd3.js  14.5 kB       0
           1.a6a4935a84d4fc7fddd3.js  23.1 kB       1
         app.a6a4935a84d4fc7fddd3.js  1.46 MB       2       app
Child html-webpack-plugin for "index.html":
                                   Asset     Size  Chunks       Chunk Names
    13ed6d2a0f1addfda9c8530374554c10.png    12 kB
                              index.html  1.46 MB       0
webpack: bundle is now VALID.
invalid
{ '0': '/Users/fcarasso/Dev/console-proto/src/main.js',
  '1': 1476804803000 }
webpack: bundle is now INVALID.
invalidAsync
{ '0':
   Watching {
     startTime: 1476804803016,
     invalid: false,
     error: null,
     stats:
      Stats {
        compilation: [Object],
        hash: 'a6a4935a84d4fc7fddd3',
        startTime: 1476804802571,
        endTime: 1476804802791 },
     handler: [Function],
     watchOptions: { aggregateTimeout: 200 },
     compiler:
      Compiler {
        _plugins: [Object],
        outputPath: '/Users/fcarasso/Dev/console-proto/dist',
        outputFileSystem: [Object],
        inputFileSystem: [Object],
        recordsInputPath: undefined,
        recordsOutputPath: undefined,
        records: [Object],
        fileTimestamps: [Object],
        contextTimestamps: [Object],
        resolvers: [Object],
        parser: [Object],
        options: [Object],
        context: '/Users/fcarasso/Dev/console-proto',
        name: 'client',
        watchFileSystem: [Object],
        _lastCompilationFileDependencies: [Object],
        _lastCompilationContextDependencies: [] },
     running: true,
     watcher: null },
  '1': [Function: next] }
invalid
{}
webpack building...
webpack built a6a4935a84d4fc7fddd3 in 207ms
Hash: a6a4935a84d4fc7fddd3
Version: webpack 2.1.0-beta.22
Time: 207ms
                               Asset     Size  Chunks       Chunk Names
13ed6d2a0f1addfda9c8530374554c10.png    12 kB
007abab9d8f8ae99042650e9311bdc82.png  16.1 kB
7f86a225ce5b3f8879ef567e8ef37743.png  14.2 kB
           0.a6a4935a84d4fc7fddd3.js  14.5 kB       0
           1.a6a4935a84d4fc7fddd3.js  23.1 kB       1
         app.a6a4935a84d4fc7fddd3.js  1.46 MB       2       app
Child html-webpack-plugin for "index.html":
                                   Asset     Size  Chunks       Chunk Names
    13ed6d2a0f1addfda9c8530374554c10.png    12 kB
                              index.html  1.46 MB       0
webpack: bundle is now VALID.
Tue 18 Oct 2016 11:33:23 EDT
Tue 18 Oct 2016 11:33:24 EDT
Tue 18 Oct 2016 11:33:25 EDT
^C

[mischkl]

We are experiencing the endless rebuilding problem even for files that weren't created shortly before watching started. This is a problem that has affected us with Webpack 1 as well as 2, and occurs sporadically but often enough that it makes working with webpack-dev-server a chore and it almost makes one long for the days of grunt-watch and BrowserSync. ;) Here is a link to our config:
https://gist.github.com/mischkl/cf7ab4b10df9763aa698a5b885f700cd

This is on Windows 7 Enterprise x64 with Node 6.9.1, and seems to primarily happen when IntelliJ IDEA is open.

I should also note that we have seen the problem when using angular-cli with the default settings as well. The fact that it only seems to affect a subset of users makes me strongly suspect that this is a Windows/Intellij/WebStorm-related issue.