README: bring expectation of canonical-data.json values (description, errors, no msg) up-to-date#551
README: bring expectation of canonical-data.json values (description, errors, no msg) up-to-date#551petertseng merged 3 commits intoexercism:masterfrom petertseng:readme-desc
Conversation
petertseng
changed the title
|
Previous discussion of descriptions: #451 (comment) |
| - the description should not simply explain **what** each case is (that is redundant information) | ||
| - the description should explain **why** each case is there | ||
| - for example, what kinds of incorrect implementations might this case help us find? | ||
| - 'variable names': one or more variable names with values which will be passed to the solution method |
There was a problem hiding this comment.
I was wondering about the "will be passed to the solution method" part. Shouldn't that be "will be passed to the test method"?
There was a problem hiding this comment.
so I guess here "solution method" means "method in the solution". In leap, we'd get "year":1996 telling us to do is_leap 1996 or isLeap(1996) or similar. I do think "solution method" makes more sense than "test method". Basically I think it's to make clear that it's what the student wrote and is submitting.
If it still bears improvement, maybe I could offer "function under test", or simply "the submitted function" (maybe too specific, if not all submissions are single functions). I would have also offered "the student's function" but that canonicalises the student word that I use, whereas other people use others (practitioner, exercist, user...)
There was a problem hiding this comment.
Ah I see. Good point then.
|
This PR seems like a great improvement. |
| - expected: the expected result (this would be -1 when we expect an exception) | ||
| - msg: a nice message for the failing case | ||
| - expected: the expected result | ||
| - if an error is expected, the value at `"expected"` should be an object containing exactly one property, `"error"`, whose value is a string. |
There was a problem hiding this comment.
In this proposal, how should one differentiate between a missing value (optional return) and an error? Is it possible, or both should use the same structure?
There was a problem hiding this comment.
This is also a point of attention to me. In #336 I suggest to use null as the value to represent a missing value. Languages with support for optional types can then use their optional type instead of null.
There was a problem hiding this comment.
Ah, I rubber-ducked myself into finding a reasonable answer. I've added the possibility of null, on the assumption that my guess is correct. I'm very open to corrections, of course.
| - description: which will be used to name each generated test | ||
| - the description should not simply explain **what** each case is (that is redundant information) | ||
| - the description should explain **why** each case is there | ||
| - for example, what kinds of incorrect implementations might this case help us find? |
There was a problem hiding this comment.
This line seems to belong to the line above (about the why). Should it really be on a new line?
There was a problem hiding this comment.
you're right, there is no good reason to put it on another
|
I think this looks great now! |
| - expected: the expected result (this would be -1 when we expect an exception) | ||
| - msg: a nice message for the failing case | ||
| - expected: the expected result | ||
| - if the input is valid but there is no result for the input, the value at `"expected"` should be `null`. |
There was a problem hiding this comment.
While the last proposal in #336 doesn't enforce the existence of a expected property, if it is present, it can now also be null.
|
I think we can merge this now, right? |
3 participants
Note that while there is a new schema proposed in #336, it's believed by me that everything being said in this PR is still applicable even when that schema is applied:
So this may be evaluated without having to wait for #336