[Ansor] Parallel the InitPopulation

[jcf94]

Some small fix:

• Add a read-write lock for SplitFactorizationMemo
• Parallel the InitPopulation
• Fix the warning of test_auto_scheduler_layout_rewrite test

@FrozenGene I tested this parallel implementation in my server, the running time of tests/python/unittest/test_auto_scheduler_search_policy.py reduced from 33s to 27s. Since your environment suffered more from this performance problem, you can have a try to see if this can help.

cc @merrymercy @comaniac

[merrymercy]

We probably should only benchmark the sample initial population part and compare the actual speedup with idea speedup.
We then can know how effective the parallelization is.

[jcf94]

We probably should only benchmark the sample initial population part and compare the actual speedup with idea speedup.
We then can know how effective the parallelization is.

Good suggestion, from the tuning log I can see it reduced from an average of 2~3 seconds:

A CUDA test from the master branch
...
Sample Initial Population       #s: 456 fail_ct: 2048   Time elapsed: 2.60

to an average of 0.05~0.5 seconds:

The same test from this branch(This even tried more populations)
...
Sample Initial Population       #s: 648 fail_ct: 3448   Time elapsed: 0.17

on my server. cc @FrozenGene

And I'm thinking about maybe the Evolutionary Search part can also benefit from parallel_for in the same way? @merrymercy @comaniac

[comaniac]

LGTM. Yeah I think the mutation part could also use parallel_for.