Hidden rests from <forward> tags not filled with rests by xmlToM21

[jacobtylerwalls]

I have an xml source with hidden rests implied from forward tags in Voice 2, and the import parsing leaves me with gaps if the forward tag occurs midmeasure. See the example below: Voice 2 consists of forward/note/forward. The result of parsing Voice 2 yields rest, note instead of rest, note, rest.

music21/music21/musicxml/xmlToM21.py

Line 2317 in 74b3428

v.makeRests(inPlace=True, hideRests=True)

Is there a reason fillGaps is not set to True here? Is it so that the "filling" of incomplete final measures is avoided?

I can loop through every measure with voices and call makeRests(fillGaps=True, hideRests=True) on every voice, but I am curious why xmlToM21 is making its current assumptions.

Example measure tag attached
hiddenRestsVoice.txt

[mscuthbert]

Is this the output produced by Finale? I'm curious that this behavior hasn't appeared before. I'm not sure that the behavior is intentional.

If you could edit out the file to the minimum possible code that reproduces the unexpected behavior, it'd help me to get this in sooner -- that is, removing all the formatting information (line thickness, etc.) to just keep the part of the file that I can build a test around. Bug fixing includes adding tests to ensure that the bug does not appear again. Thanks!

[jacobtylerwalls]

Minimal example now attached. Yes, the original file was Finale export. Before investing more effort I wanted to see if it sounded intentional -- the reason being that it looks like there was code in xmlForward() to create these rests, but it was commented out in the last release. I haven't tested the old version.

hiddenRestsVoice_minimal.txt

[jacobtylerwalls]

The file in my previous comment had some schema validation issues (working a bit too fast, sorry!), fixed here (Github doesn't support uploading .xml)

hiddenRestsVoice_minimal_2.txt
:

[mscuthbert]

Perfect! I'll be able to work with this on the next hack session I have -- this is a very simple test case. Thanks!

[jacobtylerwalls]

I looked more closely at how Finale handles hidden rests. Even in a single-voice scenario, Finale will handle a hidden rest as a forward tag. So in a minimal example consisting of one hidden rest and one visible rest, XMLToM21 picks up the visible rest only.

from xml.etree.ElementTree import fromstring as EL
measure = '<measure><forward><duration>5040</duration>
</forward><note><rest /><duration>5040</duration></note></measure>'
mxMeasure = EL(measure)
mp = musicxml.xmlToM21.MeasureParser(mxMeasure)
mp.parse()
mp.stream.show('text')

OUT: {0.5} <music21.note.Rest rest>

My original example showed xmlToM21 creating only leading rests in multiple-voice scenarios but not trailing rests. This is because makeRests() is only called if the parser identifies multiple voices and makeRests() only creates rests before the offset on the element it is called on, which in the original example, was the note appearing midmeasure.

I see two solutions for supporting hidden rests from forward tags: either calling makeRests(fillGaps=True) whether or not self.useVoices(), which is what my program is doing, or more surgically, restoring the code I see you drafted in this commit (which then required more work to not break a Bach corpus example):
944785d

[jacobtylerwalls]

Multiple voice scenarios are the most realistic use case, so sticking with that for now.

Found a way to catch these without causing problems with incomplete final measures.

Summary: leading hidden rests were being found because makeRests() creates them in all cases, but mid-measure and end of measure hidden rests from elements were not created.

fillGaps handles the midmeasure case, and using refStreamOrTimeRange to compare with the entire measure stream handles the end case.

Adjusted the test file to include an instance of a midmeasure hidden rest so that it includes hidden rests in all three locations.