Skip to content

Comments

Update assert_writeln_magic to the latest libdparse#2758

Merged
dlang-bot merged 3 commits intodlang:masterfrom
Geod24:magic
Aug 4, 2020
Merged

Update assert_writeln_magic to the latest libdparse#2758
dlang-bot merged 3 commits intodlang:masterfrom
Geod24:magic

Conversation

@Geod24
Copy link
Member

@Geod24 Geod24 commented Feb 19, 2020

I noticed on the CI that this was throwing a lot of errors.
Turns out it was using a libdparse from late 2017,
which didn't knew about a lot of syntax changes,
such as 'body' -> 'do', new contract syntax, extern(C++, string)...

The two small changes required were introduced by:

@Geod24 Geod24 requested a review from wilzbach as a code owner February 19, 2020 11:35
@dlang-bot
Copy link
Contributor

Thanks for your pull request, @Geod24!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

@Geod24
Copy link
Member Author

Geod24 commented Feb 19, 2020

For reference I see 2,131 parsing errors on a recent build, in the form:

magic.d(2330:5)[error]: Primary expression expected
magic.d(2330:64)[warn]: Empty declaration
magic.d(4564:33)[error]: Basic type, type constructor, symbol, or typeof expected
magic.d(4564:74)[warn]: Empty declaration
magic.d(4584:13)[error]: Expected `}` instead of `else`
magic.d(4585:53)[warn]: Empty declaration
magic.d(4607:35)[error]: Basic type, type constructor, symbol, or typeof expected
magic.d(4607:60)[error]: Declaration expected
magic.d(4607:86)[error]: Declaration expected
magic.d(4607:87)[error]: Declaration expected
magic.d(4607:88)[error]: Declaration expected
magic.d(4609:21)[error]: Declaration expected
magic.d(4610:61)[error]: Declaration expected
magic.d(4610:75)[error]: Declaration expected
magic.d(4610:76)[warn]: Empty declaration
magic.d(4612:17)[error]: Declaration expected
magic.d(4612:64)[error]: Declaration expected
magic.d(4615:17)[error]: Declaration expected
magic.d(4615:64)[error]: Declaration expected
magic.d(4618:17)[error]: Declaration expected
magic.d(4621:64)[error]: Declaration expected
magic.d(4621:65)[error]: Declaration expected
magic.d(4623:29)[error]: Declaration expected
magic.d(4623:30)[warn]: Empty declaration
magic.d(4624:13)[error]: Declaration expected
magic.d(4625:13)[error]: Declaration expected
magic.d(4627:9)[error]: Declaration expected
magic.d(4645:5)[error]: Declaration expected
magic.d(4646:5)[error]: Declaration expected
magic.d(4647:49)[warn]: Empty declaration
magic.d(4648:1)[error]: Declaration expected
magic.d(4682:5)[error]: Primary expression expected
magic.d(4682:47)[warn]: Empty declaration
magic.d(5561:21)[error]: Basic type, type constructor, symbol, or typeof expected
magic.d(5561:46)[warn]: Empty declaration
magic.d(8629:28)[error]: Basic type, type constructor, symbol, or typeof expected
magic.d(8629:64)[error]: Declaration expected
magic.d(8629:65)[warn]: Empty declaration
magic.d(8652:5)[error]: Declaration expected
magic.d(8653:1)[error]: Declaration expected

@Geod24
Copy link
Member Author

Geod24 commented Feb 19, 2020

Blocked by dlang-community/libdparse#383

@Geod24
Copy link
Member Author

Geod24 commented Jun 25, 2020

Great, this is now blocked on an update to ddox from libdparse v0.8.x to v0.15.1

So that the shebang can be used.
Geod24 added 2 commits August 4, 2020 09:28
I noticed on the CI that this was throwing a lot of errors.
Turns out it was using a libdparse from late 2017,
which didn't knew about a lot of syntax changes,
such as 'body' -> 'do', new contract syntax, extern(C++, string)...

The two small changes required were introduced by:
- dlang-community/libdparse@daa93a5#diff-277d73641e4843decd9c9a41c6c23046L820
- dlang-community/libdparse@e625d2b#diff-ad149e8970c93d472185f0fb76ffcaeaR86
@Geod24
Copy link
Member Author

Geod24 commented Aug 4, 2020

After almost 6 months, this is finally ready to go 🎉

There are still a few issues showing up, but much less than there used to be, so yay?

find .generated/phobos-2.093.0/etc .generated/phobos-2.093.0/std -name '*.d' | \
	sed -e /unittest.d/d | sort >> .generated/.latest-files.txt
.generated/dmd-2.093.0/generated/linux/release/64/dmd -J.generated/dmd-2.093.0/src/dmd/res -J.generated/dmd-2.093.0/res -J.generated/dmd-2.093.0/generated/linux/release/64/ -c -o- -version=CoreDdoc \
	-version=MARS -version=CoreDdoc -version=StdDdoc -Df.generated/.latest-dummy.html \
	-Xf.generated/docs-latest.json -I.generated/phobos-2.093.0 @.generated/.latest-files.txt
dpl-docs/dpl-docs filter .generated/docs-latest.json --min-protection=Protected \
	--only-documented --ex=core.internal --ex=core.stdc.config --ex=core.sys --ex=std.algorithm.internal --ex=std.c --ex=std.internal --ex=std.regex.internal --ex=std.digest.digest --ex=std.windows.registry --ex=etc.linux.memoryerror --ex=std.typetuple --ex=msvc_dmc --ex=msvc_lib --ex=dmd.libmach --ex=dmd.libmscoff --ex=dmd.scanmach --ex=dmd.scanmscoff --ex=dmd.libmach --ex=dmd.libmscoff --ex=dmd.scanmach --ex=dmd.scanmscoff
magic.d(9:39)[error]: Declaration expected
magic.d(76:1)[error]: Declaration expected
magic.d(15:1)[error]: Declaration expected
magic.d(16:1)[error]: Declaration expected
magic.d(8:1)[error]: Declaration expected
magic.d(8:1)[error]: Declaration expected
magic.d(7:1)[error]: Declaration expected
magic.d(24:1)[error]: Declaration expected
magic.d(11:1)[error]: Declaration expected
magic.d(11:1)[error]: Declaration expected
magic.d(17:1)[error]: Declaration expected
magic.d(6:1)[error]: Declaration expected
magic.d(25:27)[error]: Declaration expected
magic.d(25:60)[error]: no identifier for declarator
magic.d(25:62)[error]: declaration expected instead of `{`
magic.d(26:1)[error]: Declaration expected
magic.d(16:1)[error]: Expected `)` instead of `EOF`
magic.d(16:1)[error]: type expected
magic.d(21:1)[error]: Declaration expected
magic.d(8:1)[error]: Declaration expected
magic.d(15:1)[error]: Declaration expected
Reading doc file...
Parsing JSON...
Filtering modules...
No name for module .generated/dmd-2.093.0/src/dmd/backend/os.d - ignoring
No name for module .generated/dmd-2.093.0/src/dmd/backend/filespec.d - ignoring
No name for module .generated/dmd-2.093.0/src/dmd/backend/divcoeff.d - ignoring
No name for module .generated/dmd-2.093.0/src/dmd/backend/compress.d - ignoring
No name for module .generated/dmd-2.093.0/src/build.d - ignoring
No name for module .generated/druntime-2.093.0/src/test_runner.d - ignoring
No name for module .generated/druntime-2.093.0/src/rt/invariant.d - ignoring
Writing filtered docs...
rm -f .generated/.latest-files.txt .generated/.latest-dummy.html
dpl-docs/dpl-docs generate-html --file-name-style=lowerUnderscored --std-macros=html.ddoc --std-macros=dlang.org.ddoc --std-macros=std.ddoc --std-macros=macros.ddoc --std-macros=std-ddox.ddoc \
	--override-macros=std-ddox-override.ddoc --package-order=std \
	--git-target=v2.093.0 --no-exact-source-links \
	.generated/docs-latest.json /dev/shm/dtest/work/repo/dlang.org/web/library
No type found for dmd.parse.Parser.Parser.NeedDeclaratorId.
Error parsing type 'const(void function()* function() pure nothrow @property @nogc)': Missing ')' for 'const('
Function object.ModuleInfo.tlsctor has non-function type: Primitive
Error parsing type 'const(void function()* function() pure nothrow @property @nogc)': Missing ')' for 'const('
Function object.ModuleInfo.tlsdtor has non-function type: Primitive
Error parsing type 'const(void function()* function() pure nothrow @property @nogc)': Missing ')' for 'const('
Function object.ModuleInfo.ctor has non-function type: Primitive
Error parsing type 'const(void function()* function() pure nothrow @property @nogc)': Missing ')' for 'const('
Function object.ModuleInfo.dtor has non-function type: Primitive
Error parsing type 'const(void function()* function() pure nothrow @property @nogc)': Missing ')' for 'const('
Function object.ModuleInfo.ictor has non-function type: Primitive
Error parsing type 'const(void function()* function() pure nothrow @property @nogc)': Missing ')' for 'const('
Function object.ModuleInfo.unitTest has non-function type: Primitive
Error parsing type 'const extern (C++) void function(pointer ptr)': Expected identifier but got C++
Function core.stdcpp.memory.default_delete.default_delete.opCall.opCall has non-function type: Primitive
Error parsing type 'extern (C++) extern (C++) void function() nothrow @nogc* function(extern (C++) void function() nothrow @nogc*) nothrow @nogc': Expected identifier but got C++
Function core.stdcpp.exception.set_terminate has non-function type: Primitive
Error parsing type 'extern (C++) extern (C++) void function() nothrow @nogc* function() nothrow @nogc': Expected identifier but got C++
Function core.stdcpp.exception.get_terminate has non-function type: Primitive
Error parsing type 'extern (C++) extern (C++) void function() @nogc* function(extern (C++) void function() @nogc*) nothrow @nogc': Expected identifier but got C++
Function core.stdcpp.exception.set_unexpected has non-function type: Primitive
Error parsing type 'extern (C++) extern (C++) void function() @nogc* function() nothrow @nogc': Expected identifier but got C++
Function core.stdcpp.exception.get_unexpected has non-function type: Primitive
Error parsing type '(T[N] args...)': Expecting ',', got ...
Function core.stdcpp.array.array.array.this has non-function type: Primitive
Error parsing type 'const(std.range.chain!(std.algorithm.iteration.joiner!(std.algorithm.iteration.MapResult!(std.bitmanip.BitArray.bitsSet().__lambda2, std.algorithm.iteration.FilterResult!(std.bitmanip.BitArray.bitsSet().__lambda1, std.range.iota!(ulong, ulong).iota(ulong, ulong).Result).FilterResult).MapResult).joiner(std.algorithm.iteration.MapResult!(std.bitmanip.BitArray.bitsSet().__lambda2, std.algorithm.iteration.FilterResult!(std.bitmanip.BitArray.bitsSet().__lambda1, std.range.iota!(ulong, ulong).iota(ulong, ulong).Result).FilterResult).MapResult).Result, std.algorithm.iteration.FilterResult!(std.bitmanip.BitArray.bitsSet().__lambda3, std.range.iota!(ulong, const(ulong)).iota(ulong, const(ulong)).Result).FilterResult).chain(std.algorithm.iteration.joiner!(std.algorithm.iteration.MapResult!(std.bitmanip.BitArray.bitsSet().__lambda2, std.algorithm.iteration.FilterResult!(std.bitmanip.BitArray.bitsSet().__lambda1, std.range.iota!(ulong, ulong).iota(ulong, ulong).Result).FilterResult).MapResult).joiner(std.algorithm.iteration.MapResult!(std.bitmanip.BitArray.bitsSet().__lambda2, std.algorithm.iteration.FilterResult!(std.bitmanip.BitArray.bitsSet().__lambda1, std.range.iota!(ulong, ulong).iota(ulong, ulong).Result).FilterResult).MapResult).Result, std.algorithm.iteration.FilterResult!(std.bitmanip.BitArray.bitsSet().__lambda3, std.range.iota!(ulong, const(ulong)).iota(ulong, const(ulong)).Result).FilterResult).Result function() pure nothrow @property)': Expecting ',', got Result
Function std.bitmanip.BitArray.bitsSet has non-function type: Primitive
Error parsing type '(U[] values...)': Expecting ',', got ...
Function std.container.array.Array.Array.this.this has non-function type: Primitive
Error parsing type '(U[] values...)': Expecting ',', got ...
Function std.container.dlist.DList.DList.this.this has non-function type: Primitive
Error parsing type '(Elem[] elems...)': Expecting ',', got ...
Function std.container.rbtree.RedBlackTree.RedBlackTree.this has non-function type: Primitive
Error parsing type '(E[] elems...)': Expecting ',', got ...
Function std.container.rbtree.redBlackTree.redBlackTree has non-function type: Primitive
Error parsing type '(E[] elems...)': Expecting ',', got ...
Function std.container.rbtree.redBlackTree.redBlackTree has non-function type: Primitive
Error parsing type '(E[] elems...)': Expecting ',', got ...
Function std.container.rbtree.redBlackTree.redBlackTree has non-function type: Primitive
Error parsing type '(E[] elems...)': Expecting ',', got ...
Function std.container.rbtree.redBlackTree.redBlackTree has non-function type: Primitive
Error parsing type '(U[] values...)': Expecting ',', got ...
Function std.container.slist.SList.SList.this.this has non-function type: Primitive
Error parsing type '(T[] items...)': Expecting ',', got ...
Function std.container.util.make.make.make has non-function type: Primitive
Error parsing type 'const(std.datetime.date.Date.isoWeekAndYear().ISOWeekAndYear function() pure nothrow @property @safe)': Missing ')' for 'const('
Function std.datetime.date.Date.isoWeekAndYear has non-function type: Primitive
Error parsing type 'pure nothrow @nogc @trusted void function(scope const(ubyte)[] data...)': Expecting ',', got ...
Function std.digest.crc.CRC.CRC.put has non-function type: Primitive
Error parsing type 'ref HMAC!(H, blockSize) function(in ubyte[] data...) return': Expecting ',', got ...
Function std.digest.hmac.HMAC.HMAC.put has non-function type: Primitive
Error parsing type 'pure nothrow @nogc void function(scope const(ubyte[]) data...)': Expecting ',', got ...
Function std.digest.murmurhash.MurmurHash3.MurmurHash3.putRemainder has non-function type: Primitive
Error parsing type 'pure nothrow @nogc void function(scope const(Element[]) elements...)': Expecting ',', got ...
Function std.digest.murmurhash.MurmurHash3.MurmurHash3.putElements has non-function type: Primitive
Error parsing type 'pure nothrow void function(scope const(ubyte)[] data...)': Expecting ',', got ...
Function std.digest.murmurhash.MurmurHash3.MurmurHash3.put has non-function type: Primitive
Error parsing type 'nothrow @trusted void function(scope const(ubyte)[] data...)': Expecting ',', got ...
Function std.digest.WrapperDigest.WrapperDigest.put has non-function type: Primitive
Error parsing type 'pure nothrow @nogc @trusted void function(scope const(ubyte)[] input...)': Expecting ',', got ...
Function std.digest.sha.SHA.SHA.put has non-function type: Primitive
Error parsing type 'Allocator[(max + 1 - min) / step]': Expecting ',', got 1
Error parsing type '(auto ref Allocator alloc, size_t[N] lengths...)': Expecting ',', got ...
Function std.experimental.allocator.makeMultidimensionalArray.makeMultidimensionalArray has non-function type: Primitive
Error parsing type 'void function(const(char)[][] recipients...)': Expecting ',', got ...
Function std.net.curl.SMTP.mailTo.mailTo has non-function type: Primitive
Error parsing type 'pure nothrow @safe immutable(C)[] function(const(C)[][] paths...)': Expecting ',', got ...
Function std.path.buildPath.buildPath has non-function type: Primitive
Error parsing type 'pure nothrow @safe immutable(C)[] function(const(C[])[] paths...)': Expecting ',', got ...
Function std.path.buildNormalizedPath.buildNormalizedPath has non-function type: Primitive
Error parsing type 'size_t function(ref Rng rnd, Num[] proportions...)': Expecting ',', got ...
Function std.random.dice.dice has non-function type: Primitive
Error parsing type 'size_t function(Num[] proportions...)': Expecting ',', got ...
Function std.random.dice.dice has non-function type: Primitive
Error parsing type '(R input, String[] re...)': Expecting ',', got ...
Function std.regex.matchFirst.matchFirst has non-function type: Primitive
Error parsing type '(R input, String[] re...)': Expecting ',', got ...
Function std.regex.matchAll.matchAll has non-function type: Primitive
Error parsing type '@property T function(inout __InoutWorkaroundStruct = __InoutWorkaroundStruct.init)': Expecting ',', got __InoutWorkaroundStruct
Function std.traits.rvalueOf.rvalueOf has non-function type: Primitive
Error parsing type '@property ref T function(inout __InoutWorkaroundStruct = __InoutWorkaroundStruct.init)': Expecting ',', got __InoutWorkaroundStruct
Function std.traits.lvalueOf.lvalueOf has non-function type: Primitive
Error parsing type 'string function(const char[][] names...)': Expecting ',', got ...
Function std.typecons.alignForSize.alignForSize has non-function type: Primitive
Error parsing type '(uint[] intervals...)': Expecting ',', got ...
Function std.uni.InversionList.InversionList.this.this has non-function type: Primitive
Error parsing type '(scope const C[] chars...)': Expecting ',', got ...
Function std.uni.Grapheme.this.this has non-function type: Primitive
Reading doc file...
Parsing JSON...
Parsing docs...
Finished parsing docs.

Additionally, a lot of what I think are false positive because of this:

.generated/ddoc_preprocessor --compiler=../dmd/generated/linux/release/64/dmd -conf= -Df.generated/dlangspec.html macros.ddoc html.ddoc dlang.org.ddoc doc.ddoc .generated/2.093.0.ddoc nodatetime.ddoc .generated/dblog_latest.ddoc ebook.ddoc .generated/dlangspec.d

I don't think libdparse handles ddoc correctly.
And finally, the following from dpl-docs itself:

Skipping external navigation item: https://wiki.dlang.org/Get_involved
Warning: Item in navigation does not exist: foundation/contributors.html
Warning: Item in navigation does not exist: foundation/index.html
Warning: Item in navigation does not exist: foundation/donate.html
Warning: Item in navigation does not exist: foundation/sponsors.html
Skipping external navigation item: https://tour.dlang.org
Skipping external navigation item: https://wiki.dlang.org/Books
Skipping external navigation item: https://wiki.dlang.org/Tutorials
Skipping external navigation item: https://wiki.dlang.org/Development_tools
Skipping external navigation item: https://wiki.dlang.org/Editors
Skipping external navigation item: https://wiki.dlang.org/IDEs
Skipping external navigation item: https://run.dlang.io
Skipping external navigation item: http://rainers.github.io/visuald/visuald/StartPage.html
Warning: Unknown page: /changelog/2.078.0.html
  (linked from 1 pages e.g. changelog/2.090.0.html)
Warning: Unknown page: /spec/cpp_interface.html
  (linked from 1 pages e.g. changelog/2.091.0.html)
Warning: Unknown page: abi.html
  (linked from 1 pages e.g. dlangspec.html)
Warning: Unknown page: arrays.html
  (linked from 1 pages e.g. spec/simd.html)
Warning: Unknown page: articles.html
  (linked from 1 pages e.g. index.html)

@dlang-bot dlang-bot merged commit 5d52249 into dlang:master Aug 4, 2020
@Geod24 Geod24 deleted the magic branch August 4, 2020 03:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants