convert docstring examples to unittests

[wilzbach]

Inspired by #4039 I propose to go through all old modules and replace the examples in docstring with checked unittests.
The advantage is quite clear - it is possible to verify that the documentation is correct and up-to-date.

A quick regex yielded that there a 55 files affected:

107 ./datetime.d
45 ./xml.d
40 ./net/curl.d
25 ./experimental/logger/core.d
16 ./stdio.d
11 ./file.d
10 ./socket.d
10 ./parallelism.d
9 ./exception.d
7 ./regex/package.d
7 ./range/package.d
7 ./process.d
6 ./net/isemail.d
6 ./base64.d
5 ./uni.d
5 ./random.d
5 ./functional.d
5 ./concurrency.d
4 ./windows/registry.d
4 ./traits.d
4 ./conv.d
3 ./typecons.d
3 ./path.d
3 ./internal/scopebuffer.d
3 ./experimental/allocator/typed.d
3 ./encoding.d
3 ./digest/ripemd.d
3 ./digest/digest.d
2 ./zip.d
2 ./format.d
2 ./experimental/ndslice/slice.d
2 ./experimental/logger/filelogger.d
2 ./experimental/allocator/building_blocks/stats_collector.d
2 ./digest/md.d
2 ./bitmanip.d
2 ./algorithm/setops.d
1 ./windows/syserror.d
1 ./uuid.d
1 ./string.d
1 ./socketstream.d
1 ./signals.d
1 ./math.d
1 ./internal/cstring.d
1 ./experimental/ndslice/package.d
1 ./experimental/logger/multilogger.d
1 ./experimental/allocator/building_blocks/segregator.d
1 ./digest/sha.d
1 ./demangle.d
1 ./csv.d
1 ./container/slist.d
1 ./container/rbtree.d
1 ./array.d
1 ./algorithm/sorting.d
1 ./algorithm/searching.d
1 ./algorithm/package.d

I used this command - in case you want to reproduce it:

for f in $(find . -name '*.d'); do echo $(grep "Example:" $f | wc -l) "$f"; done | grep -v "^0" | sort -nr

Questions

I would be happy to start with this task and go through the modules - maybe I can even automate it.
So there are only two questions

1. If there is a global module example (like in this PR for uuid), currently the best thing is to add the unittest below the class and add the magic \\\. However this means that the example will be added to the class and not the module. How about using \\\\ for module unittest examples?
   Compare http://dlang.org/phobos/std_uuid.html and the generated documentation for this PR.

2. How "okay" are you with this proposal?
   It's a bit of work and it would feel like wasted time if this ends up non-merged.

I would split the PR in multiple parts, so that it's easier to review ;-)

PS/off-topic: Is anyone taking bets how many docstring examples don't work anymore?

[PetarKirov]

Hi @greenify!

1. Thank you for doing this!
2. Thanks to @quickfur 's good work, now we can put ddoc-ed unittests at module level (before all members) and they will appear in the documentation. I suggest you try this for this PR, so we can verify this on the documentation autotester.
3. Have a great day!

[wilzbach]

now we can put ddoc-ed unittests at module level (before all members) and they will appear in the documentation. I suggest you try this for this PR, so we can verify this on the documentation autotester.

It does work on local machine & the autotester - that's exactly what I was looking for :)
I would say that it makes sense to put it between module and the import statements?

Thank you for doing this!

I would go ahead once I get the okay from an official member - pinging randomly @andralex, @JackStouffer, @schveiguy, @burner

[jmdavis]

I would say that it makes sense to put it between module and the import statements?

Personally, I think that it would be a bit cleaner to put the unittest block after the imports, but I'm not sure that it matters much.

As for turning ddoc examples into ddoc-ed unittest blocks. You really don't need permission for that. It's slowly been being done over time. It's just that it's most frequently happening either a PR happens to touch some code that didn't use a ddoc-ed unittest block. Just keep in mind that there are examples that won't work as ddoc-ed unit tests. std.net.curl is a great example of this because of the network stuff that it's doing, and a bunch of the examples in std.datetime shouldn't be converted either, because the functions in question are member functions on templated types, and it's problematic to put unittest blocks inside of templates. Also, in some rare cases, the documentation purposefully has an example in the middle of it, in which case, turning the example into a ddoc-ed unittest block would mess up the documentation. So, don't just automatically convert an example to a ddoc-ed unit test without thinking about it. I expect that that majority of examples that aren't ddoc-ed unittest blocks should be. So, there should be plenty for you to do on that front, but not every example should be converted.

So, feel free to create some PRs which turn examples into ddoc-ed unittest blocks where appropriate, but please make sure that the PRs are of a manageable size - e.g. there's no need to try and fix every example in one PR. If you do it in smaller chunks, it'll be easier to review and verify the changes.

[wilzbach]

So, feel free to create some PRs which turn examples into ddoc-ed unittest blocks where appropriate, but please make sure that the PRs are of a manageable size

Understood - I regard this PR as completed now.
It turns four examples into unittests, the questionable case is signals.d, but here I hope observedMessageCounter makes it still obvious what is going on.

As a side remark: I moved also the unittests for SList #4039, DList #4040 and RBTree #4041 to the top ;-)

[schveiguy]

If you do it in smaller chunks, it'll be easier to review and verify the changes

As a further incentive, PRs that affect less files are less likely to encounter merge conflicts.

[wilzbach]

I even found a case where the docstring isn't displayed: experimental/allocator/building_blocks/stats_collector.d

See:

http://dlang.org/phobos/std_experimental_allocator_building_blocks_stats_collector.html
https://github.com/D-Programming-Language/phobos/blob/master/std/experimental/allocator/building_blocks/stats_collector.d#L195

[DmitryOlshansky]

LGTM

[DmitryOlshansky]

Auto-merge toggled on