Dramatically faster to compile Tuple

[andralex]

Investigation following discussion in #5916

[dlang-bot]

Thanks for your pull request, @andralex!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

[wilzbach]

How about:

static foreach (i, val; fieldSpecs)
{{
     alias name = val.name;

[andralex]

@wilzbach done

[quickfur]

Wow. Yeah, std.format is a bear in CTFE. It's pretty awesome that you can actually use it in CTFE, but performance is thrown out the window.

[wilzbach]

Btw another huge slow-down is parseSpecs above. When I use staticMap which is invalid, but allows the code to compile, it gets from 0.05s further down to 0.01s. So how do improve parseSpecs?

template parseSpecs(Specs...)
{
    static if (Specs.length == 0)
    {
        alias parseSpecs = AliasSeq!();
    }
    else static if (is(Specs[0]))
    {
        // SWAP this with the block below -> _a lot_ faster 
        // alias parseSpecs = staticMap!FieldSpec;
        static if (is(typeof(Specs[1]) : string))
        {
            alias parseSpecs =
                AliasSeq!(FieldSpec!(Specs[0 .. 2]),
                          parseSpecs!(Specs[2 .. $]));
        }
        else
        {
            alias parseSpecs =
                AliasSeq!(FieldSpec!(Specs[0]),
                          parseSpecs!(Specs[1 .. $]));
        }
    }
    else
    {
        static assert(0, "Attempted to instantiate Tuple with an "
                        ~"invalid argument: "~ Specs[0].stringof);
    }
}

[wilzbach]

The auto-tester failures look weird 😦

[wilzbach]

Before this change:

module	bare import time
std.regex	0.92s
std.net.curl	0.28s
std.concurrency	0.24s
std.encoding	0.18s
std.zip	0.15s
std.file	0.15s
std.datetime	0.15s
std.path	0.14s
std.mmfile	0.14s
std.socket	0.12s
std.string	0.10s
std.uni	0.09s
std.bigint	0.05s
std.process	0.05s
std.zlib	0.04s
std.getopt	0.04s
std.numeric	0.04s
std.bitmanip	0.04s
std.algorithm	0.04s
std.parallelism	0.04s
std.uuid	0.03s
std.range	0.03s
std.utf	0.02s
std.math	0.02s
std.json	0.02s
std.conv	0.02s
std.stdio	0.02s
std.variant	0.02s
std.signals	0.02s
std.complex	0.02s
std.typecons	0.02s
std.container	0.02s
std.mathspecial	0.02s
std.uri	0.01s
std.csv	0.01s
std.meta	0.01s
std.array	0.01s
std.traits	0.01s
std.stdint	0.01s
std.random	0.01s
std.format	0.01s
std.digest	0.01s
std.base64	0.01s
std.exception	0.01s
std.functional	0.01s
std.xml	0.00s
std.ascii	0.00s
std.system	0.00s
std.demangle	0.00s
std.compiler	0.00s
std.outbuffer	0.00s

With this change:

module	bare import time
std.regex	0.94s
std.net.curl	0.28s
std.zip	0.17s
std.path	0.15s
std.file	0.14s
std.mmfile	0.14s
std.datetime	0.14s
std.socket	0.13s
std.encoding	0.12s
std.string	0.11s
std.uni	0.09s
std.concurrency	0.08s
std.process	0.06s
std.zlib	0.04s
std.getopt	0.04s
std.bigint	0.04s
std.numeric	0.04s
std.algorithm	0.04s
std.uuid	0.03s
std.range	0.03s
std.bitmanip	0.03s
std.parallelism	0.03s
std.utf	0.02s
std.json	0.02s
std.stdio	0.02s
std.variant	0.02s
std.signals	0.02s
std.complex	0.02s
std.typecons	0.02s
std.container	0.02s
std.mathspecial	0.02s
std.uri	0.01s
std.csv	0.01s
std.meta	0.01s
std.math	0.01s
std.conv	0.01s
std.array	0.01s
std.traits	0.01s
std.stdint	0.01s
std.random	0.01s
std.format	0.01s
std.digest	0.01s
std.base64	0.01s
std.exception	0.01s
std.functional	0.01s
std.xml	0.00s
std.ascii	0.00s
std.system	0.00s
std.demangle	0.00s
std.compiler	0.00s
std.outbuffer	0.00s

Look at e.g. std.concurrency :)

Script used

[andralex]

@wilzbach could the autotester failures have something to do with the recent changes in the test infra?

[wilzbach]

@wilzbach could the autotester failures have something to do with the recent changes in the test infra?

This PR (dlang/dmd#7444) - submitted an hour ago - is passing on all machines:

https://auto-tester.puremagic.com/pull-history.ghtml?projectid=1&repoid=1&pullid=7444

[quickfur]

@wilzbach Re: parseSpecs, it's hard to say how to improve it. Maybe some form of static foreach might work? Not sure how, though. Maybe in the form of a CTFE function + mixin, something along the lines of:

string impl()
{
    string result = "AliasSeq!(";
    static foreach (spec; specs)
    {
        result ~= ... /* whatever transformation we want here */
    }
    result ~= ")";
    return result;
}
alias x = mixin(impl());

But I've no idea whether the cost of CTFE + mixin is actually any better than just expanding the current template implementation. I suppose for very large specs it might surpass the template expansion, due to less symbols being generated, but that's pure speculation on my part. For small specs, it's a toss-up which one is better. Have to profile to know for sure.

[quickfur]

P.S. Well, the idea behind using static foreach is so that CTFE doesn't actually run the loop, but is just concatenating a bunch of string snippets. Still, it's hard to say whether it will outperform template expansion.

[andralex]

How do I restart the tester? I tried git push -f myfork faster-tuple , no avail.

[wilzbach]

could the autotester failures have something to do with the recent changes in the test infra?

So I can reproduce the failures locally when trying to build the DMD testsuite with Phobos built with this branch :/

[andralex]

@wilzbach what command(s) do you run? The things I tried work on my system.

[wilzbach]

@wilzbach what command(s) do you run? The things I tried work on my system.

Are your dmd and druntime up to date?
It's simply:

cd ~/dlang/dmd/test && make clean && make

[andralex]

@wilzbach eh, I now see it works. What gives?

[wilzbach]

You just reverted the commit ...

[andralex]

Oh OK. It seems the import of std.conv was the culprit, likely resulting in exposing a CT bug. I've replaced to!string with stringof which is better anyway, and left the bug discovery to another fight.

[andralex]

On my Airtop i7 with the latest: before (best of 5): 0.286s, after: 0.038s.

[wilzbach]

@andralex: you seem to have amended your work to the revert commit. I rebased your PR and squashed both commits into one.

[UplinkCoder]

@andralex proofs my point about templates.

staticMap was the culprit here.