feat: enable e2e orchestrator restart by gartnera · Pull Request #2264 · zeta-chain/node

gartnera · 2024-05-24T17:50:52Z

Description

Enable restarting the e2e orchestrator by:

always use the --setup-only option followed by running with the --skip-setup option
skip setup if the deployed.yml (created by setup) exists

This enables you to rerun the e2e tests just by running docker restart orchestrator

Also fix the TestDappZEVM/TestDappEVM addresses. Otherwise you'll see something like this:

Details

setup      | ✅ setup completed in 32.235828535s
setup      | ✅ config file written in /work/deployed.yml
setup      |  --- 📜zEVM contracts ---
setup      | SystemContract: 0x91d18e54DAf4F677cB28167158d6dd21F6aB3921
setup      | ETHZRC20:       0x13A0c5930C028511Dc02665E7285134B6d11A5f4
setup      | ERC20ZRC20:     0x48f80608B672DC30DC7e3dbBd0343c5F02C738Eb
setup      | BTCZRC20:       0xd97B1de3619ed2c6BEb3860147E30cA8A7dC9891
setup      | UniswapFactory: 0x9fd96203f7b22bCF72d9DCb40ff98302376cE09c
setup      | UniswapRouter:  0x2ca7d64A7EFE2D62A725E2B35Cf7230D6677FfEe
setup      | ConnectorZEVM:  0x239e96c8f17C85c30100AC26F635Ea15f23E9c67
setup      | WZeta:          0x5F0b1a82749cb4E2278EC87F8BF6B618dC71a8bf
setup      | ZEVMSwapApp:    0x733aB8b06DDDEf27Eaa72294B0d7c9cEF7f12db9
setup      | ContextApp:     0xD28D6A0b8189305551a0A8bd247a6ECa9CE781Ca
setup      | TestDappZEVM:       0xA8D5060feb6B456e886F023709A2795373691E63
setup      |  --- 📜EVM contracts ---
setup      | ZetaEth:        0xcd01737eaa83195514601E9F6df929Bd0709DC60
setup      | ConnectorEth:   0xfdA26637E11771E5919aF5c24A363c82613d62dA
setup      | ERC20Custody:   0x03Ce5CDd0aEC673B11a672df176E0F52E9191556
setup      | ERC20:      0x486d325e1D577f1fe6f4E1633E137f91758a8362
setup      | TestDappEVM:       0x28b64929f082F7361B9d9e681AaC3436dA4Cb857
setup      | ✅ the localnet has been setup
running e2e tests...
setup      | starting E2E tests
setup      | ⚙️ setting up TSS address
setup      | ⚙️ enabling verification flags for block headers
setup      |  --- 📜zEVM contracts ---
setup      | SystemContract: 0x91d18e54DAf4F677cB28167158d6dd21F6aB3921
setup      | ETHZRC20:       0x13A0c5930C028511Dc02665E7285134B6d11A5f4
setup      | ERC20ZRC20:     0x48f80608B672DC30DC7e3dbBd0343c5F02C738Eb
setup      | BTCZRC20:       0xd97B1de3619ed2c6BEb3860147E30cA8A7dC9891
setup      | UniswapFactory: 0x9fd96203f7b22bCF72d9DCb40ff98302376cE09c
setup      | UniswapRouter:  0x2ca7d64A7EFE2D62A725E2B35Cf7230D6677FfEe
setup      | ConnectorZEVM:  0x239e96c8f17C85c30100AC26F635Ea15f23E9c67
setup      | WZeta:          0x5F0b1a82749cb4E2278EC87F8BF6B618dC71a8bf
setup      | ZEVMSwapApp:    0x733aB8b06DDDEf27Eaa72294B0d7c9cEF7f12db9
setup      | ContextApp:     0xD28D6A0b8189305551a0A8bd247a6ECa9CE781Ca
setup      | TestDappZEVM:       0x0000000000000000000000000000000000000000
setup      |  --- 📜EVM contracts ---
setup      | ZetaEth:        0xcd01737eaa83195514601E9F6df929Bd0709DC60
setup      | ConnectorEth:   0xfdA26637E11771E5919aF5c24A363c82613d62dA
setup      | ERC20Custody:   0x03Ce5CDd0aEC673B11a672df176E0F52E9191556
setup      | ERC20:      0x486d325e1D577f1fe6f4E1633E137f91758a8362
setup      | TestDappEVM:       0x28b64929f082F7361B9d9e681AaC3436dA4Cb857

<snip>

setup      | ❌ zevm mp panic: waiting cctx timeout, cctx not mined, inboundHash: 0x900d53080f3c437a9b34d6d8214a5ddfee2ba4eacc32fb4c8ec63b925d2bc9dc, stack trace goroutine 103 [running]:
github.com/zeta-chain/zetacore/cmd/zetae2e/local.zevmMPTestRoutine.func1.1()
	/go/delivery/zeta-node/cmd/zetae2e/local/zevm_mp.go:30 +0x60
panic({0x233b660, 0x4001d7ffd0})
	/usr/local/go/src/runtime/panic.go:884 +0x1f4
github.com/zeta-chain/zetacore/e2e/utils.WaitCctxsMinedByInboundHash({0x35084f8, 0x400004e780}, {0x4000edc4b0, 0x42}, {0x3530778, 0x400060dd20}, 0x1, {0x34e1f00, 0x40004c7e90}, 0x0)
	/go/delivery/zeta-node/e2e/utils/zetacore.go:62 +0x6bc
github.com/zeta-chain/zetacore/e2e/utils.WaitCctxMinedByInboundHash({0x35084f8?, 0x400004e780?}, {0x4000edc4b0, 0x42}, {0x3530778?, 0x400060dd20?}, {0x34e1f00?, 0x40004c7e90?}, 0xde0b6b3a7640000?)
	/go/delivery/zeta-node/e2e/utils/zetacore.go:28 +0x4c
github.com/zeta-chain/zetacore/e2e/runner.(*E2ERunner).WaitForMinedCCTX(0x400100aa00, {0x90, 0xd, 0x53, 0x8, 0xf, 0x3c, 0x43, 0x7a, 0x9b, ...})
	/go/delivery/zeta-node/e2e/runner/zeta.go:36 +0x1b0
github.com/zeta-chain/zetacore/cmd/zetae2e/local.zevmMPTestRoutine.func1()
	/go/delivery/zeta-node/cmd/zetae2e/local/zevm_mp.go:58 +0x248
golang.org/x/sync/errgroup.(*Group).Go.func1()
	/go/pkg/mod/golang.org/x/sync@v0.5.0/errgroup/errgroup.go:75 +0x5c
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/pkg/mod/golang.org/x/sync@v0.5.0/errgroup/errgroup.go:72 +0x9c

setup      | ❌ e2e tests failed after 6m26.432842915s

BTC tests are failing on the restart, but this is a good first step:

Details

skipping e2e setup because it has already been completed
running e2e tests...
setup      | starting E2E tests
setup      | ⚙️ setting up TSS address
setup      | ⚙️ enabling verification flags for block headers
setup      |  --- 📜zEVM contracts ---
setup      | SystemContract: 0x91d18e54DAf4F677cB28167158d6dd21F6aB3921
setup      | ETHZRC20:       0x13A0c5930C028511Dc02665E7285134B6d11A5f4
setup      | ERC20ZRC20:     0x48f80608B672DC30DC7e3dbBd0343c5F02C738Eb
setup      | BTCZRC20:       0xd97B1de3619ed2c6BEb3860147E30cA8A7dC9891
setup      | UniswapFactory: 0x9fd96203f7b22bCF72d9DCb40ff98302376cE09c
setup      | UniswapRouter:  0x2ca7d64A7EFE2D62A725E2B35Cf7230D6677FfEe
setup      | ConnectorZEVM:  0x239e96c8f17C85c30100AC26F635Ea15f23E9c67
setup      | WZeta:          0x5F0b1a82749cb4E2278EC87F8BF6B618dC71a8bf
setup      | ZEVMSwapApp:    0x733aB8b06DDDEf27Eaa72294B0d7c9cEF7f12db9
setup      | ContextApp:     0xD28D6A0b8189305551a0A8bd247a6ECa9CE781Ca
setup      | TestDappZEVM:       0xA8D5060feb6B456e886F023709A2795373691E63
setup      |  --- 📜EVM contracts ---
setup      | ZetaEth:        0x733aB8b06DDDEf27Eaa72294B0d7c9cEF7f12db9
setup      | ConnectorEth:   0xD28D6A0b8189305551a0A8bd247a6ECa9CE781Ca
setup      | ERC20Custody:   0xff3135df4F2775f4091b81f4c7B6359CfA07862a
setup      | ERC20:      0xbD1e64A22B9F92D9Ce81aA9B4b0fFacd80215564
setup      | TestDappEVM:       0xBFF76e77D56B3C1202107f059425D56f0AEF87Ed

<snip>

zevm_mp    | ✅ completed in 41.117951979s - evm -> zevm message passing contract with failing revert
zevm_mp    | 🍾 ZEVM message passing tests completed in 5m1.017731008s
erc20      | ✅ completed in 12.073586308s - deposit ERC20 into ZEVM restricted address
erc20      | 🍾 erc20 tests completed in 5m3.565783507s
ether      | ✅ completed in 44.112256664s - deposit ZRC20 into ZEVM and call a contract
ether      | ⏳running - deposit Ether into ZEVM and call a contract that reverts; should refund
ether      | ✅ completed in 1m44.285673884s - deposit Ether into ZEVM and call a contract that reverts; should refund
ether      | ⏳running - withdraw Ether from ZEVM to restricted address
zeta       | ✅ completed in 2m30.333301442s - message passing with successful revert between external EVM chains
zeta       | ⏳running - deposit ZETA from Ethereum to ZEVM restricted address
zeta       | ✅ completed in 7.049840791s - deposit ZETA from Ethereum to ZEVM restricted address
zeta       | ⏳running - deposit ZETA from Ethereum to ZEVM
zeta       | ✅ completed in 13.081878008s - deposit ZETA from Ethereum to ZEVM
zeta       | ⏳running - deposit ZETA from Ethereum to a new ZEVM address which does not exist yet
ether      | ✅ completed in 24.081093927s - withdraw Ether from ZEVM to restricted address
ether      | 🍾 Ethereum tests completed in 7m11.938481337s
zeta       | ✅ completed in 11.072837997s - deposit ZETA from Ethereum to a new ZEVM address which does not exist yet
zeta       | 🍾 Zeta tests completed in 7m21.337073437s
bitcoin    | ✅ BTC deposited in 6m59.602731686s
setup      | ❌ bitcoin panic: timed out waiting for block header to show up in observer, stack trace goroutine 46 [running]:
github.com/zeta-chain/zetacore/cmd/zetae2e/local.bitcoinTestRoutine.func1.1()
	/go/delivery/zeta-node/cmd/zetae2e/local/bitcoin.go:32 +0x60
panic({0x233b780, 0x34daba0})
	/usr/local/go/src/runtime/panic.go:884 +0x1f4
github.com/zeta-chain/zetacore/e2e/runner.(*E2ERunner).ProveBTCTransaction(0x40014af100, 0x4001b016a0)
	/go/delivery/zeta-node/e2e/runner/bitcoin.go:425 +0x72c
github.com/zeta-chain/zetacore/e2e/runner.(*E2ERunner).DepositBTC(0x40014af100, 0x1)
	/go/delivery/zeta-node/e2e/runner/bitcoin.go:181 +0x430
github.com/zeta-chain/zetacore/cmd/zetae2e/local.bitcoinTestRoutine.func1()
	/go/delivery/zeta-node/cmd/zetae2e/local/bitcoin.go:65 +0x290
golang.org/x/sync/errgroup.(*Group).Go.func1()
	/go/pkg/mod/golang.org/x/sync@v0.5.0/errgroup/errgroup.go:75 +0x5c
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/pkg/mod/golang.org/x/sync@v0.5.0/errgroup/errgroup.go:72 +0x9c

setup      | ❌ e2e tests failed after 7m30.243773648s
e2e failed

Closes: #2232

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Include instructions and any relevant details so others can reproduce.

feat: enable e2e orchestrator restart

6383f2d

gartnera added the no-changelog Skip changelog CI check label May 24, 2024

gartnera requested review from brewmaster012, kingpinXD, lumtis, skosito and ws4charlie as code owners May 24, 2024 17:50

github-actions bot added the breaking:cli label May 24, 2024

gartnera mentioned this pull request May 24, 2024

fix: try fixing E2E crosschain_swap failure #2266

Merged

14 tasks

lumtis approved these changes May 27, 2024

View reviewed changes

skosito approved these changes May 27, 2024

View reviewed changes

Merge branch 'develop' into enable-e2e-orchestrator-restart

50419fc

lumtis merged commit 8144a09 into develop May 28, 2024

lumtis deleted the enable-e2e-orchestrator-restart branch May 28, 2024 07:32

gartnera mentioned this pull request May 28, 2024

fix: remove quotes from e2e args #2284

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: enable e2e orchestrator restart#2264

feat: enable e2e orchestrator restart#2264
lumtis merged 2 commits intodevelopfrom
enable-e2e-orchestrator-restart

gartnera commented May 24, 2024 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

gartnera commented May 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How Has This Been Tested?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gartnera commented May 24, 2024 •

edited

Loading