As discussed in haskell/hsc2hs#25 there are various services on Windows that can keep a file open for a fraction of time, such as indexer or AV services. That means that deleteFile may fail temporarily.
It's a reasonably common idiom, SQLlite for instance does so as well https://www.sqlite.org/src/info/89f1848d7f and we do it in the GHC testsuite and now hsc2hs to avoid a race condition with the file deletion.
Would it be perhaps handy to have such a function in Directory?