Fix: Problem Builder does not show on Progress page until student submits an answer (OC-1033)

[bradenmacdonald]

OC-1033

Fix Description
Without this fix, Problem Builder blocks do not appear on a student's "Progress" page until the student has submitted a grade. The reason was that Problem Builder did not implement the max_score() method.

Potential Further Improvement
Problem Builder blocks currently return scores as floating point values between 0 and 1. It would be better to return integer numerator/denominator pairs (simpler for students, more reliable, more consistent with CAPA modules). However, changing that in the middle of any live course could cause some students' scores to be weighted differently than others. (e.g. Student A could have a score of '0.5/1' while student B could submit the same answer post-upgrade and get a score of '1/2' - which is theoretically the same but would be weighted more heavily when computing the grade of that unit as a whole).

That said, if the currently live course[s] mostly use problem builder blocks containing only one question each, it would make sense to make that change/fix now.

Note
This PR also includes some fixes for the Travis CI build, but those changes do not affect the code itself and are contained to .travis.yml

[bradenmacdonald]

@antoviaque Any thoughts on the "Potential Further Improvement" described above?

[antoviaque]

Good question - I would be fine with the change if the only impact is the problem weight, I think the other courses using it would be able to accomodate it; but I'm not sure, for the Davidson course we are considering, if this would be acceptable. @dseaton What do you think? (Or who else should we ask from Davidson about this?)

[dseaton]

Because we don't have strict grading of problem-builder (PB) problems, it actually benefits the students to have these problems weighted more heavily. Additionally, I tried to put 1PB per block, but others did not abide by this standard - so it will be a mix. My feeling is that an update shouldn't matter because the weighting would only benefit students - they only need to write "anything" to get credit. Before proceeding, can someone confirm my logic?

I want to note that there are two issues:

1. As Braden said, problem builder does not show progress until an attempt occurs.
2. We have students who have completed all problems in a sequence, but are receiving 0.5 credit on problem builder problems.

Are we also addressing 2?

[bradenmacdonald]

@antoviaque Ok, great, glad the change would be ok for Davidson - but is Harvard or anyone else also currently using this block on edx.org ?

@dseaton I'm still trying to diagnose/reproduce issue 2. The change I've suggested could help but I'm not yet certain.

[Kelketek]

👍 Works for me.

[dseaton]

@bradenmacdonald Not sure about other schools using problem builder. Will need to wait for @antoviaque

[bradenmacdonald]

@antoviaque When you're answering my question above, note that the change would be ported to master as well, so would also affect clients using the latest version and not just the edx-release version.

[antoviaque]

@bradenmacdonald For Harvard it should be fine - they do use it but the exercises aren't graded. For the impact on McKinsey they are still upgrading from v1 to v2, so since this doesn't affect the v1 blocks we should be fine too. The only area I can think of that could be problematic would be for the datawise team, if there was an impact on the dashboard (the one with the color coding), but I wouldn't expect it?

One issue though is that there might be other courses, which aren't our clients, which could be using problem builder, either on edx.org or open edx instances. If that was the case, would course teams be able to fix it/make it consistent by triggering a regrading of the questions from the instructor dashboard? If so, imho that should be fine - we could just warn about it on the mailing list to be friendly to other people that might be using it.

[bradenmacdonald]

@antoviaque Ok. I will do that in a second follow-up PR, to keep this one contained so we can make the release target.

[adampalay]

👍

[bradenmacdonald]

Following up on the discussion here about further improvements, it seems that the platform always scales the score of our block to the value of its weight field. (So whether we report the score as "0.8" or "4/5", students always see just "0.8" if the weight is kept at its defaults value of 1.0) So although that means that we could actually re-scale the scores to be integers rather than floats without any consequences, it also means that there wouldn't be much point in doing so. So I'm going to just leave that as-is.