backport: merge bitcoin#23046, #24138, #24812, #24626, #21726, #25123, #25074, #24832, #26215, #24858, #26417, #16981 (index backports)

[kwvg]

Additional Information

• When backporting bitcoin#23046, it was discovered that there has been a longstanding bug in CDeterministicMNManager::MigrateDBIfNeeded(2)() that flagged a database taken from an older version for failing its "previous migration attempt", requiring the database to be fully rebuilt through a reindex.

  This occurred because the older database would be read pre-DIP3 in MigrateDBIfNeeded(), which then caused the migration logic to write the new best block (source) (the legacy best block is erased before the DIP3 condition is checked, source) but while it completed the transaction (source), critically, it didn't write it to disk (example of writing to disk, here).

  This meant that when it was read again by MigrateDBIfNeeded2(), it saw three things a) there is no new best block (because it didn't get written), b) there is no legacy best block (because it gets erased before the new best block is written) and c) that the chain height is greater than 1 (since this isn't a new datadir and the chain has already advanced), it concludes that it was a botched migration attempt and fails (source).

  This bug affects v19 to develop (3f0c2ff3 as of this writing) and prevents feature_txindex_compatibility.py from working as expected as it would migrate legacy databases to newer versions to test txindex migration code and get stuck at unhappy EvoDB migration logic, to allow the test to function properly when testing against the latest version of the client, this bug has been fixed as part of this PR.

• In bitcoin#23046, version v0.17 was used as the last version to support legacy txindex as the updated txindex format was introduced in dash#4178 (i.e. after v0.17) and the version selected for having migration code in it (note, migration code was removed in dash#6296, so far not included as part of any release) was v18.2.2 despite the range being v18.x to v21.x was a) due to the bug mentioned above affecting v19.x onwards and b) v18.2.2 being the latest release in the v18.x lifecycle.

  • The specific version number used for v0.17 is 170003 as the binaries corresponding to 170000 are not populated in releases/, which causes a CI failure (source)

• As of develop (3f0c2ff3 as of this writing), feature_pruning.py was broken due to changes in Core that were not adjusted for, namely:

  • The enforcement of MAX_STANDARD_TX_SIZE (source) from DIP1 onwards (source) resulting in bad-txns-oversize errors in blocks generated for the test as the transactions generated are ~9.5x larger than the now-enforced limit (source), this is resolved by pushing the DIP1 activation height upwards to 2000 (the same activation height used for DIP3 and DIP8).
  • Change in subsidy logic in v20 (source) that results in bad-cb-amount errors, this is resolved by pushing the v20 activation height upwards.

  Additionally, an inopportune implicit post-generate sync (source) also causes the test to fail. Alongside the above, they have been resolved in this PR.

• As of develop (3f0c2ff3 as of this writing), bench_dash crashes when running the AssembleBlock benchmark. The regression was traced back to bitcoin#21840 (5d10b41) in dash#6152 due to the switch to P2SH_OP_TRUE.

  This has been resolved by reverting this particular change.

  Pre-fix test failure:

  $ ./src/bench/bench_dash
  Warning, results might be unstable:
  * CPU governor is '' but should be 'performance'
  * Turbo is enabled, CPU frequency will fluctuate
  
  Recommendations
  * Use 'pyperf system tune' before benchmarking. See https://github.com/psf/pyperf
  
  |               ns/op |                op/s |    err% |          ins/op |         bra/op |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|---------------:|--------:|----------:|:----------
  |       17,647,657.00 |               56.66 |    0.1% |  231,718,349.00 |  42,246,265.00 |    0.1% |      0.20 | `AddrManAdd`
  |       42,201,861.00 |               23.70 |    0.1% |  544,366,811.00 | 102,569,244.00 |    0.0% |      0.46 | `AddrManAddThenGood`
  |          189,697.83 |            5,271.54 |    0.1% |    1,763,991.40 |     356,189.40 |    0.3% |      0.01 | `AddrManGetAddr`
  |              454.38 |        2,200,808.04 |    0.6% |        6,229.11 |       1,343.92 |    0.1% |      0.01 | `AddrManSelect`
  |        1,066,471.00 |              937.67 |   67.6% |   13,350,463.00 |   3,150,465.00 |    0.4% |      0.01 | :wavy_dash: `AddrManSelectByNetwork` (Unstable with ~1.0 iters. Increase `minEpochIterations` to e.g. 10)
  |        1,181,774.50 |              846.19 |   49.0% |   18,358,489.50 |   4,224,642.50 |    0.0% |      0.02 | :wavy_dash: `AddrManSelectFromAlmostEmpty` (Unstable with ~1.1 iters. Increase `minEpochIterations` to e.g. 11)
  bench_dash: bench/block_assemble.cpp:46: void AssembleBlock(benchmark::Bench &): Assertion `res.m_result_type == MempoolAcceptResult::ResultType::VALID' failed.
  [1]    2343746 IOT instruction (core dumped)  ./src/bench/bench_dash
  

Breaking changes

None expected

Checklist

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added or updated relevant unit/integration/functional/e2e tests
• I have made corresponding changes to the documentation (note: N/A)
• I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

[UdjinM6]

utACK 7d9ff96

[PastaPastaPasta]

utACK 7d9ff96