Patch AnswerHints

[taniwallach]

Patch AnswerHints so that if a subroutine in the list "dies" there will still be a final result from the AnswerHints postfilter.
This is mean to address #679.

[drgrice1]

I suppose this is okay.

I have mixed feelings about this though.

Your example code in the linked issue should set $tmp = 0 and then wrap the $tmp = $student->eval(x => 0) in a perl eval. Then the answerHint macro will work as expected and give the message that the condition requested in the problem is not satisfied. This pull request will hide this error, and make it harder to see that the hints are not working in the intended way.

On the other hand, it doesn't hurt to catch errors from special cases don't matter that are accidentally missed by the author.

[pstaabp]

Fixes the error in the posted problem. I would recommend the cleanups that @drgrice1 pointed out.

[taniwallach]

@drgrice1 @pstaabp - Thanks. Perltidy changes are in the third commit.

Your example code in the linked issue should set $tmp = 0 and then wrap the $tmp = $student->eval(x => 0) in a perl eval. Then the answerHint macro will work as expected and give the message that the condition requested in the problem is not satisfied.

Thanks. I fixed the real problem (was more detailed, but the core error was the occurrence of a divide by zero error for a student answer at the initial condition point). I had not considered the need to such until this occurred.

I thought it would be helpful to document this in the Wiki, but the Wiki problems technique page about AnswerHints does not even mention the use of a subroutine test, so explaining there that evaluation at a test point could potentially lead to such an issue seems overboard there.

This pull request will hide this error, and make it harder to see that the hints are not working in the intended way.

Good point. I added a fixed "error message" (as a result table message) for when the eval fails. That will make the existence of a problem visible to the end user (who may report it) but prevents the post filter from dying and messing up the allocation of weighed credit, etc.

I tried unsuccessfully to use the value of $@ to retrieve the internal error message ("Divide by zero"). Using warn ref($@); could get that to show up as a warning message at the bottom, but I did not manage to get it out to put into the results table. It seems my understanding of what a failed eval is putting into $@ is just not sufficient to dig it out. Ideas for how to do that are welcome.

[drgrice1]

The perltidy result is a little different with the remove options added back in, but it will do for now.

I am not sure that I agree with displaying the "error during AnswerHints processing" as an answer message. I think of those messages as informative messages for the student. I think it might be better if that were a simple warn statement so that it appears with the information about reporting the issue to the professor and such. This is really an issue with the problem. In any case, this is will do for now also.

I am not sure why $@ is not being set exactly. It most likely has to do with the $SIG{__WARN__} handling inside the translator though. The way that you dealt with it seems to be working well though.

[pstaabp]

This looks good now. I agree with @drgrice1 about the message. We could say something along the lines of "An error occurred in this problem. Please inform your instructor."

[taniwallach]

This looks good now. I agree with @drgrice1 about the message. We could say something along the lines of "An error occurred in this problem. Please inform your instructor."

Sounds good to me, but I would like to somehow mention AnswerHints in the message, to provide a hint to whoever debugs it of where to look. How about: "An error occurred in this problem. Please inform your instructor. (die in AnswerHints subroutine)"?

[drgrice1]

I found an issue with the eval approach. See my comment.

[taniwallach]

I am not sure that I agree with displaying the "error during AnswerHints processing" as an answer message. I think of those messages as informative messages for the student. I think it might be better if that were a simple warn statement so that it appears with the information about reporting the issue to the professor and such.

Upon additional consideration, I changed the error message to a warn and the text to "An error occurred in this problem."

Since it is now a warn it will trigger the standard message above the problem: "Warning -- There may be something wrong with this question. Please inform your instructor including the warning messages below." Since the warn is from inside answerHints.pl the message below the problem provides an indication to the staff where to look for the bug.