Find out why attempt.aggs.stages.inputRecords.sum is zero, even if records were processed

[kevinwallimann]

In spark history, Find out why attempt.aggs.stages.inputRecords.sum shows zero, even if QueryProgressEvent has non-zero numInputRows, e.g. like here:

{"Event":"org.apache.spark.sql.streaming.StreamingQueryListener$QueryProgressEvent","progress":{"id":"57114e13-db5e-4dc6-930b-a7ff99c77666","runId":"c71422cb-0787-4423-bf1b-491b19e83e35","name":null,"timestamp":"2022-05-26T09:04:09.740Z","batchId":39556,"durationMs":{"triggerExecution":9331,"getEndOffset":0,"queryPlanning":254,"getBatch":8,"setOffsetRange":908,"addBatch":7738,"walCommit":195},"eventTime":{},"stateOperators":[],"sources":[{"description":"KafkaV2[Subscribe[redacted.topic]]","startOffset":"{"redacted.topic":{"2":172810,"5":172921,"4":172097,"1":172469,"3":172141,"0":172512}}","endOffset":"{"redacted.topic":{"2":172826,"5":172933,"4":172112,"1":172479,"3":172154,"0":172524}}","numInputRows":78,"inputRowsPerSecond":"NaN","processedRowsPerSecond":8.359232665309184}],"sink":

[kevinwallimann]

Required for issue: Example of attempt.aggs.stages.inputRecords.sum showing zero, exact job to reproduce the issue