SerialPorts Linux issues

[krwq]

Collective issue for all issues while work is still happening.
Most of the issues were identified when working on async implementation: dotnet/corefx#33027

There are still lots of failures, all of them are marked with custom [KnownFailure] - most of the tests won't run on CI because it doesn't have physical port.

Things known to still be broken on Linux:

• (Issue dotnet/corefx#33374; PR System.IO.Ports native package corefx#35126) System.IO.Ports package cannot be used with netcoreapp2.1 even though it depends on netstandard2.0 (requires 3.0)
• software emulated RTS (XonXoff) is not working (or at least all related tests are failing)
• There are issues with parity replacement - likely just doesn't work
• There are issues with DTREnable - those could be test issues
• BytesToWrite (looks ok through code inspection but seem to be misreporting, possibly data from the queue should be added there)
• SerialPort.ReadBufferSize is not respected (see below conversation)
• in certain cases DataReceived event can fire more than once (SerialStream does it correctly but code on top of that in SerialPort is not) - this is likely also issue on Windows - I've added a comment why this happens how this can be fixed on Linux (likely something similar can be done on Windows) - we need to explore if this is something we want to fix
• Some tests related to StopBits are failing when measuring the transmission time - this is likely a test issue but needs to be looked at to double check
• non standard BaudRates are currently not supported but I've seen some article explaining how to emulate those - there are also couple of test failures (or rather hangs) but likely those are test issues (except for when non-standard baud rate is used) - currently allowed baud rate ranges are hard coded on linux (couldn't find an easy way to get that info from OS - perhaps just trying to set it and seeing if OS allows it is a way to go)
  • (Method group allocates when it shouldn't #33747) High baud rates - I've removed the artificial limit
  • (allow to set custom serial speed on Linux #80534) Custom baud rates
• flushing is not working (currently no-op)
• BytesToRead may return -1 when error occurs which is causing APIs such SerialPort.ReadLine throw ArgumentException in some cases where there are errors during communication
• [test] HasHardwareFlowControl is not working correctly - this can be checked similarly as RTS related tests are doing it (this might have a broader meaning than just RTS/CTS so needs more look)
• (Running CoreCLR tests on Mono - Update the issues.target file with Git hub links as new issues are created for each failing excluded test #33258) Uart port is not always detected
• (ConcurrentDictionary.GetOrAdd is not always thread safe. #33221) Dispose implementation (i.e. Shutdown call might need to be moved to a new SafeHandle class)
  • exclusive access need to be removed explicitly since some devices do not get rid of it making device not openable after closing

[krwq]

Moving this to future since I've addressed issues which I found useful to fix. Remainder is stuff which is very rarely used and we can address as we need.

Please upvote/comment on issues if you find this and want to have something sooner so that we can prioritize accordingly.

[ss4adam]

Setting SerialPort.ReadBufferSize property doesn't seem to affect the Linux tty buffer:

_serialPort.ReadBufferSize = 3000000; // three megabytes, this works fine in windows and lets me read a massive chunk of data, but in Linux I can't seem to read anything past 4096 bytes.

later if I use _serialPort.Read(dataBuffer, 0, dataBuffer.Length); then I miss anything outside of this 4K range. I like reading a large buffer rather than lots of small reads because I'm sending files and worried that the computer freezing up could make me lose a piece of the data in the middle of the transfer

I know in the past the linux kernel had an internal buffer of 4096 bytes.
If this buffer is full and a new character arrives on the serial port,
the oldest character in the buffer will be overwritten and thus will be lost. I'm using RedHat7

Is this a problem that .NET can even fix if the kernel is limiting the buffer size? I'm not strong at Linux so I'm not sure. If this can't be fixed, should an exception be thrown?

[krwq]

Hi @ss4adam, yes, this is fixable (assuming kernel allows you to do so) although we might not have cycles to fix this so any help would be appreciated (note: we will be doing repo move soon so might be better to wait until that's finished).

I'll update the list shortly

[ss4adam]

Hey @krwq, not sure what "might not have cycles to fix" means, but I would love to help. Just give me some direction so that I know where I'm looking to make the fix since I'm new to github (I'll probably need to brush up on com development). Another solution I was thinking about in the case that tty buffer might not be dynamically changeable (I looked at some linux source and saw it was hardcoded in a header file N_TTY_BUF_SIZE), maybe have it just read the 4k buffer in a looping/polling manner under the covers and update bigger buffer incrementing the index as it moves along, much like normal stream reading, but then .NET users will be able to use _serialPort.ReadBufferSize = 3000000 and never know the that the linux buffer was actually never changed.

[krwq]

@ss4adam "no cycles" = no time

Thanks for offering the help!

Note corefx will not merge any PRs after tomorrow and new repo will show up which merges coreclr and corefx. I'd recommend to wait until then before making PRs.

The fix will likely be here:
https://github.com/dotnet/corefx/blob/master/src/System.IO.Ports/src/System/IO/Ports/SerialStream.Unix.cs#L373

but you can also backtrace from here:
I'd start with searching who uses this:
https://github.com/dotnet/corefx/blob/master/src/System.IO.Ports/src/System/IO/Ports/SerialPort.cs#L377

You'll likely need to add whatever syscall you need in one of those places:
https://github.com/dotnet/corefx/tree/master/src/System.IO.Ports/src/Interop/Unix
https://github.com/dotnet/corefx/tree/master/src/Common/src/Interop/Unix

[danmoseley]

@ss4adam also, general contributing info here: https://github.com/dotnet/corefx/#contributing-guide

Note, this repo will be merging into another one in a few days. Don't panic -- if you start your work against this one, it will be trivial to continue the work in the new one.

[valeriob]

Just out of curiosity, @ss4adam why do you need such a large buffer ?
What speed is the serial configured to, and why are'nt you able to keep up ?

[ss4adam]

@valeriob Each machine I use is different, the highest is usually over 300K baud, but most are 115200 (which would never have a problem with 4k buffer). I am able to keep up using BaseStream.Read() in a polling manner, but on windows environments I'm able to make the buffer bigger, which is useful if the application freezes and I miss one of my polling iterations.

[valeriob]

Thanks for explaining @ss4adam, my 2c : if you need that reliabilty you should think about doing your own polling loop in a different thread, that's how i handle it, it works fine 😄

[ss4adam]

@valeriob I actually do it it's own thread, but I'm still seeing dropped data when the machine gets overloaded with other applications, not exactly sure why yet.

[skang0401]

I have a device with a baud rate of 14400. How can I use it under Linux with system.IO.Ports

[krwq]

@skang0401 currently not supported. See note above about "Non-standard low baud rates" if you're interested in adding support

[Ellerbach]

I have a device with a baud rate of 14400. How can I use it under Linux with system.IO.Ports

@skang0401 if you need this quite fast, you can use a project I've been doing before the Linux System.IO.Ports which is perfectly working for low baud rates starting at 50. Check here: https://github.com/Ellerbach/serialapp/, there is an existing Nuget as well: https://www.nuget.org/packages/NetCoreSerial/