Scan too fast when in shm mode

[ksmith0]

After playing around with the scope some I believe that the errors reported below occur when the scan code gets ahead of the shm transmission. If I artificially slow the scope ProcessRawEvent code down the messages disappear. @cthornsb, perhaps you can comment on what is causing this?

ReadSpill: Received spill of 4052 words, but read 13160 words
ReadSpill: Spill split between buffers
ReadSpill: bad buffer, numEvents = 0
ReadSpill: bad buffer, numEvents = 0
ReadBuffer: Unexpected header length: 0
ReadBuffer:   Buffer 0 of length 13289
ReadBuffer:   CHAN:SLOT:CRATE 2:0:0
ReadSpill: Received spill of 4052 words, but read 13289 words
ReadSpill: Spill split between buffers
ReadSpill: bad buffer, numEvents = 0
ReadSpill: bad buffer, numEvents = 0
ReadSpill: bad buffer, numEvents = 0

As a side, should ProcessRawEvent be plural?

[cthornsb]

There is only one rawEvent, it is a std::deque of ChannelEvents and should not be plural. The raw "event" is a separate thing from the channel events. It is a collection of channel events which have been grouped into a time window. Maybe it would be beneficial to change the name of rawEvent to something else, for clarity.

As for the speed issue, that part of the scan code is not threaded. You build up a rawEvent and then process it. I don't see why slowing down the processing loop should fix this problem unless there is something else going on that I'm not aware of. Unpacker::ReadSpill is called first from ScanMain, then Unpacker::ReadBuffer, then ::ScanList and finally ::ProcessRawEvent. At the moment, I don't see how you could get errors from ReadSpill because the processing is going too fast. I will need to think about it more.

[ksmith0]

Ok, I understand the plural issue and we can leave that as is.

As for the speed issue, I was thinking it had to do with the fact that the scan code grabbed an incomplete spill from the UDP transfer and tried to process that. As in the scan code gets ahead of the shm broadcast and starts taking data out of UDP before it is completely flushed.

[ksmith0]

@cthornsb Any news on this?