prevent exception from race condition in tryGetReflectedDefinition #11465
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I have discovered a sort-of race condition within the FSharp.Core quotation library, that causes an "Item already exists..." exception to be thrown inside
tryGetReflectedDefinition.This PR simply prevents the exception from occurring. It does not address the problem itself, since I think it is worth further discussion. It is however causing me a problem in a production system, so I would appreciate the "band-aid" to at least prevent the exception.
The code in
tryGetReflectedDefinitionmaintains a dictionary cachereflectedDefinitionTableand also a dictionary that appears to act as a hashsetdecodedTopResources.The function syncs access to
reflectedDefinitionTableby means of a lock, and where an item is missing it proceeds to load the binary quotations for the whole assembly, unpickle and then store the results withinreflectedDefinitionTable. It also then remembers this assembly/resource has been processed by way ofdecodedTopResources.The problem occurs when more than one thread hits this code path at the same time. All threads pass the initial lock since the requested key does not exist. This then causes all threads to (wastefully) unpickle the resources for the assembly. Finally, the first thread to reach the next locked section will populate the
reflectedDefinitionTableand mark the assembly as processed viadecodedTopResources. This locked section is cognizant that other threads might have already stored what they are looking for, however it does not account for the case where the key it is looking for does not exist (which will happen many times since it is invoked through theMethodWithReflectedDefinitionpattern.) In this case, the thread will then wastefully re-add all the reflected definitions (which works since it uses the indexer) and try to re-add thedecodedTopResourcesentries, causing an exception to be thrown since it uses theAddmethod.There are various issues with the current design, but the cost of this function is tiny compared to
deserializeand friends, so it might not be worth bothering to re-work it.This change should have no impact, unless someone has a program relying on this exception in a threaded scenario ;) My current work around is to catch such errors and simply re-compile the quotations, which is far from ideal.