Expose existing FileStream.Lock() and FileStream.Unlock() APIs on MacOS/Unix

[xied75]

Dear Team,

As the subject line states we seek the consideration to add Lock() and Unlock() for FileStream in the contract. The implementation should be straight forward at least for Windows platform. Current .NET 4.61 calls Win32 LockFile underneath:
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365202(v=vs.85).aspx

Although we don't have access to LockFile for UWP, we do luckily have the brother of it LockFileEx, which also supports async, which is good news.

I believe for Linux/BSD/Mac there should be easy equivalent to find as locking on file is very common in old days. Will find all the details if the team is willing to consider the proposal.

We could do our own P/Invoke to call the LockFileEx, but it would be much elegant if corefx support that out of box, unless there is significant reason to reject it.

[stephentoub]

I believe for Linux/BSD/Mac there should be easy equivalent to find as locking on file is very common in old days. Will find all the details if the team is willing to consider the proposal.

It can be done with fcntl and F_SETLK.

[xied75]

Was testing some code found on SO: http://stackoverflow.com/questions/1784195/using-lockfileex-in-c-sharp

Using the first solution by the OP himself, the code works in normal project, once I tried it within a UWP unit test (with xunit), it gives me this:

0xC000000D was unhandled
Message: Unhandled exception at 0x77099B20 (ntdll.dll) in LockFileUWPTests.exe: 0xC000000D: An invalid parameter was passed to a service or function.

In fact calling lock() returned true, but after that if I do ReadByte() it will break.

fs.ReadByte() /// works
fs.lock(1L, 1000L) ///ok
fs.ReadByte() /// exception!

Not sure is this due to the code just set IntPtr.Zero to hEvent is the culprit, or the thread model of uwp, or even xunit runner, lacking too much knowledge.

Tried to use the 2nd answer which seems more correct to set Overlapped properly, but once within UWP I have no access to System.Threading.Overlapped class (not namespace) anymore, therefore I don't know how to do the pack() and unpack() stuff.

[joshfree]

/cc @ianhays

[terrajobst]

This would require additional APIs in .NET Framework, so I'm moving it to Future.

[stephentoub]

@terrajobst, what about:
https://msdn.microsoft.com/en-us/library/system.io.filestream.lock(v=vs.110).aspx
https://msdn.microsoft.com/en-us/library/system.io.filestream.unlock(v=vs.110).aspx
?

[terrajobst]

😄 I see. In that case RTM seems fine :-)

I don't see a problem with exposing these APIs. @KrzysztofCwalina @weshaggard?

[weshaggard]

I don't have any real context on these perhaps @ericstj might have some ideas as to why we didn't expose them in the first place.

[ayende]

They are probably hidden because they are not portable, see: http://0pointer.de/blog/projects/locking.html

[xied75]

The blog post is surprisingly negative towards Linux, makes one want to dump the penguin forever. :)

I kind of doubt that No serious software use locks in Linux, at least git does.

[ayende]

@xied75 Git doesn't use file locks. It use lock files.
In other words, it use the presence of a file to detect whatever it is lock or not, it doesn't use the flock or F_SET_LK methods.
I just grepped the git source to verify that, by the way.

[ericstj]

At the time we did the initial contract we had 3-strikes against this API.

1. Sub-file locking was not supported by all platforms. We couldn't lock the files which we got through the WinRT broker since we didn't have a handle and WinRT didn't expose anything like sub-file locking. That's not a blocker since there are other similar things we can't do with brokered files.
2. The failure mode for a partially locked file (Read/Write vs Open) is not something that folks harden against so we didn't want to promote this as the way to control access to a resource. Instead folks can either use FileShare to restrict the entire file, or coordinate region access themselves if they happen to own both consumers of the file.
3. Noone asked for the API and usage data said it was rarely used. IIRC all uses were just folks wrapping it but not actually calling it.

I don't think any of these are reason enough to not do this if we have new data, just sharing original justification.

[terrajobst]

Thanks @ericstj. My understanding is that in AppContainer a file could be lockable if it is from a location which we can access with Win32 but not lockable when it comes from a location that can only be accessed via the WinRT broker, right?

In that case it looks like the APIs will have to throw NotSupportedException as it depends on context. This tells me that FileStream would need a capability API to allow consumers to ask up front whether the particular instance supports locking, in which case I'm for postponing the APIs so that we can bring the APIs and the capability check at the same time.

[ayende]

One common use of this API in production is here:
https://github.com/apache/lucenenet/blob/master/src/Lucene.Net.Core/Store/NativeFSLockFactory.cs#L165

When we ported this to the CoreCLR we just used the native call, and since
we need to support this on Linux, we'll probably change that to use a
locking file, instead of a file lock.

*Hibernating Rhinos Ltd *

Oren Eini* l CEO l *Mobile: + 972-52-548-6969

Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811

On Tue, Feb 23, 2016 at 8:12 PM, Eric StJohn notifications@github.com
wrote:

At the time we did the initial contract we had 3-strikes against this API.

Sub-file locking was not supported by all platforms. We couldn't lock
the files which we got through the WinRT broker since we didn't have a
handle and WinRT didn't expose anything like sub-file locking. That's not a
blocker since there are other similar things we can't do with brokered
files.
2.

The failure mode for a partially locked file (Read/Write vs Open) is
not something that folks harden against so we didn't want to promote this
as the way to control access to a resource. Instead folks can either use
FileShare to restrict the entire file, or coordinate region access
themselves if they happen to own both consumers of the file.
3.

Noone asked for the API and usage data said it was rarely used. IIRC
all uses were just folks wrapping it but not actually calling it.

I don't think any of these are reason enough to not do this if we have new
data, just sharing original justification.

—
Reply to this email directly or view it on GitHub
https://github.com/dotnet/corefx/issues/5964#issuecomment-187823796.