Skip to content

chore: fixes e2e in CI#1401

Merged
stalniy merged 1 commit intomainfrom
chore/e2e-tests
May 29, 2025
Merged

chore: fixes e2e in CI#1401
stalniy merged 1 commit intomainfrom
chore/e2e-tests

Conversation

@stalniy
Copy link
Contributor

@stalniy stalniy commented May 29, 2025

Why

broken tests

What

  1. Changes in Chromium headless in Playwright v1.49 microsoft/playwright#33566
  2. added waiters with timeouts because sometimes test is too fast and it cannot connect the wallet
  3. deployment were not closed by test because test didn't wait for transaction to be successfully processed
  4. use ubuntu-22.04 instead of ubuntu-latest which is ubuntu-24.04

Summary by CodeRabbit

  • New Features

    • Added support for configuring faucet and chain API URLs in test environments.
  • Improvements

    • Increased test timeouts and improved transaction success confirmation for more reliable testing.
    • Enhanced test progress visibility with additional console output.
    • Improved wallet top-up logic to avoid unnecessary faucet requests.
    • Updated Playwright browser configuration for better compatibility.
    • Made test runner environment and package versioning more specific and stable.
  • Bug Fixes

    • Improved reliability of network and wallet connection checks during automated tests.
  • Chores

    • Updated GitHub Actions workflows for clarity and consistency.
    • Removed unnecessary caching and debugging steps from CI workflows.

@stalniy stalniy requested a review from a team as a code owner May 29, 2025 03:14
@coderabbitai
Copy link
Contributor

coderabbitai bot commented May 29, 2025

Walkthrough

This update enhances Playwright-based end-to-end testing for the deploy-web application. It adjusts test timeouts, adds runtime logging, improves wallet funding logic, updates environment configuration, and modifies Playwright and GitHub Actions setup. Several fixtures and helper functions are revised for reliability and maintainability, with no changes to exported API signatures.

Changes

File(s) Change Summary
.github/actions/console-web-ui-testing/action.yml Removed Playwright browser caching; always installs browsers; adds CI=true to E2E tests; removes debug print.
.github/workflows/console-web-release.yml Changed runner from ubuntu-latest to ubuntu-22.04; added trailing blank line in job config.
apps/deploy-web/package.json Restricted @playwright/test dependency to patch updates only (~1.52.0).
apps/deploy-web/playwright.config.ts Increased test timeout to 60s; set Chromium channel explicitly.
apps/deploy-web/tests/deploy-hello-world.spec.ts Changed timeout to 5 * 60 * 1000; added multiple console.log progress statements.
apps/deploy-web/tests/fixture/context-with-extension.ts Removed headless param/logic; closes extension page after use.
apps/deploy-web/tests/fixture/wallet-setup.ts Added getBalance helper; skips faucet call if balance > 100 AKT; uses config for faucet URL.
apps/deploy-web/tests/fixture/test-env.config.ts Added FAUCET_URL and CHAIN_API_URL to config schema with defaults.
apps/deploy-web/tests/actions/selectChainNetwork.ts Waits for networkidle after save; ignores errors from this wait.
apps/deploy-web/tests/uiState/isWalletConnected.ts Increased wait timeout for wallet connection element to 10s.
apps/deploy-web/tests/pages/DeployBasePage.tsx After signing, waits for "Transaction success" text to appear (10s timeout).

Sequence Diagram(s)

sequenceDiagram
    participant TestRunner as Playwright Test
    participant Wallet as Wallet Extension
    participant Faucet as Faucet API
    participant Blockchain as Chain API

    TestRunner->>Wallet: Open extension, connect wallet
    TestRunner->>Blockchain: getBalance(address)
    Blockchain-->>TestRunner: Return balance
    alt Balance > 100 AKT
        TestRunner->>TestRunner: Skip faucet top-up
    else Balance <= 100 AKT
        TestRunner->>Faucet: Request funds (FAUCET_URL)
        Faucet-->>TestRunner: Faucet response
    end
    TestRunner->>TestRunner: Proceed with deployment test
    TestRunner->>TestRunner: Log progress at key steps
    TestRunner->>Wallet: Sign transaction
    Wallet-->>TestRunner: Transaction signed
    TestRunner->>TestRunner: Wait for "Transaction success"
Loading

Suggested reviewers

  • ygrishajev

Poem

In the garden of code where the Playwrights dwell,
A rabbit hops in, with stories to tell.
Timeouts are longer, wallets are wise,
Console logs twinkle like stars in the skies.
With configs refreshed and caches now gone,
This bunny applauds—let the tests hop on! 🐇✨

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

npm error Exit handler never called!
npm error This is an error with npm itself. Please report this error at:
npm error https://github.com/npm/cli/issues
npm error A complete log of this run can be found in: /.npm/_logs/2025-05-29T07_09_51_757Z-debug-0.log


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 06f0d1e and 902621f.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (11)
  • .github/actions/console-web-ui-testing/action.yml (1 hunks)
  • .github/workflows/console-web-release.yml (2 hunks)
  • apps/deploy-web/package.json (1 hunks)
  • apps/deploy-web/playwright.config.ts (2 hunks)
  • apps/deploy-web/tests/actions/selectChainNetwork.ts (1 hunks)
  • apps/deploy-web/tests/deploy-hello-world.spec.ts (1 hunks)
  • apps/deploy-web/tests/fixture/context-with-extension.ts (3 hunks)
  • apps/deploy-web/tests/fixture/test-env.config.ts (1 hunks)
  • apps/deploy-web/tests/fixture/wallet-setup.ts (2 hunks)
  • apps/deploy-web/tests/pages/DeployBasePage.tsx (1 hunks)
  • apps/deploy-web/tests/uiState/isWalletConnected.ts (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • .github/workflows/console-web-release.yml
🚧 Files skipped from review as they are similar to previous changes (8)
  • apps/deploy-web/playwright.config.ts
  • apps/deploy-web/tests/actions/selectChainNetwork.ts
  • apps/deploy-web/tests/uiState/isWalletConnected.ts
  • apps/deploy-web/package.json
  • apps/deploy-web/tests/fixture/wallet-setup.ts
  • .github/actions/console-web-ui-testing/action.yml
  • apps/deploy-web/tests/deploy-hello-world.spec.ts
  • apps/deploy-web/tests/pages/DeployBasePage.tsx
🧰 Additional context used
🪛 Biome (1.9.4)
apps/deploy-web/tests/fixture/context-with-extension.ts

[error] 22-22: Unexpected empty object pattern.

(lint/correctness/noEmptyPattern)

⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: test-deploy-web-build
🔇 Additional comments (2)
apps/deploy-web/tests/fixture/context-with-extension.ts (1)

67-67: Good addition for resource cleanup.

Explicitly closing the extension page after use is a good practice that helps prevent memory leaks and improves test reliability, which aligns well with the PR's goal of stabilizing e2e tests in CI.

apps/deploy-web/tests/fixture/test-env.config.ts (1)

6-8: LGTM: Good addition of configurable URLs for test environment.

The addition of FAUCET_URL and CHAIN_API_URL with sensible default values aligns well with the PR objective to improve e2e test reliability. The default sandbox URLs provide a stable foundation for testing.

✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@stalniy stalniy force-pushed the chore/e2e-tests branch 2 times, most recently from b593dee to 844f81a Compare May 29, 2025 03:50
@codecov
Copy link

codecov bot commented May 29, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 36.39%. Comparing base (6760528) to head (902621f).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1401      +/-   ##
==========================================
- Coverage   36.39%   36.39%   -0.01%     
==========================================
  Files         818      818              
  Lines       19681    19681              
  Branches     3642     3643       +1     
==========================================
- Hits         7163     7162       -1     
+ Misses      12343    11949     -394     
- Partials      175      570     +395     
Flag Coverage Δ *Carryforward flag
api 66.29% <ø> (ø) Carriedforward from 6760528
deploy-web 15.77% <ø> (-0.01%) ⬇️
notifications 88.46% <ø> (ø) Carriedforward from 6760528
provider-proxy 80.09% <ø> (ø) Carriedforward from 6760528

*This pull request uses carry forward flags. Click here to find out more.

see 134 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b593dee and 844f81a.

📒 Files selected for processing (5)
  • .github/actions/console-web-ui-testing/action.yml (1 hunks)
  • .github/workflows/console-web-docker-build.yml (1 hunks)
  • apps/deploy-web/playwright.config.ts (2 hunks)
  • apps/deploy-web/tests/deploy-hello-world.spec.ts (1 hunks)
  • apps/deploy-web/tests/fixture/context-with-extension.ts (3 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
  • apps/deploy-web/playwright.config.ts
  • apps/deploy-web/tests/deploy-hello-world.spec.ts
  • apps/deploy-web/tests/fixture/context-with-extension.ts
  • .github/workflows/console-web-docker-build.yml
⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: test-deploy-web-build
  • GitHub Check: test-beta

@stalniy stalniy force-pushed the chore/e2e-tests branch from 844f81a to 4bc134f Compare May 29, 2025 04:24
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
apps/deploy-web/tests/fixture/context-with-extension.ts (1)

21-22: Remove unnecessary empty object pattern.

The empty object pattern is unnecessary when no parameters are being destructured. This aligns with the static analysis hint.

-  // eslint-disable-next-line no-empty-pattern
-  context: async ({}, use) => {
+  context: async (_, use) => {
🧰 Tools
🪛 Biome (1.9.4)

[error] 22-22: Unexpected empty object pattern.

(lint/correctness/noEmptyPattern)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 844f81a and 4bc134f.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (6)
  • .github/actions/console-web-ui-testing/action.yml (1 hunks)
  • .github/workflows/console-web-docker-build.yml (1 hunks)
  • apps/deploy-web/package.json (1 hunks)
  • apps/deploy-web/playwright.config.ts (2 hunks)
  • apps/deploy-web/tests/deploy-hello-world.spec.ts (1 hunks)
  • apps/deploy-web/tests/fixture/context-with-extension.ts (3 hunks)
✅ Files skipped from review due to trivial changes (1)
  • apps/deploy-web/package.json
🚧 Files skipped from review as they are similar to previous changes (4)
  • .github/actions/console-web-ui-testing/action.yml
  • apps/deploy-web/tests/deploy-hello-world.spec.ts
  • apps/deploy-web/playwright.config.ts
  • .github/workflows/console-web-docker-build.yml
🧰 Additional context used
🪛 Biome (1.9.4)
apps/deploy-web/tests/fixture/context-with-extension.ts

[error] 22-22: Unexpected empty object pattern.

(lint/correctness/noEmptyPattern)

⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: test-beta
  • GitHub Check: test-deploy-web-build
🔇 Additional comments (2)
apps/deploy-web/tests/fixture/context-with-extension.ts (2)

32-34: Good addition of CI-specific GPU flags for browser compatibility.

Adding GPU-related Chrome flags when running in CI environment is a solid approach to handle headless browser rendering issues that commonly occur in CI environments. These flags help ensure consistent test execution by using software rendering instead of hardware acceleration.


71-71: Excellent addition of explicit page cleanup.

Explicitly closing the extension page is a good practice for proper resource cleanup and preventing potential memory leaks in test environments, especially important for long-running test suites.

@stalniy stalniy force-pushed the chore/e2e-tests branch from 4bc134f to 4c2eacd Compare May 29, 2025 04:36
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
apps/deploy-web/tests/fixture/context-with-extension.ts (1)

32-34: Consider uncommenting GPU flags for CI stability.

The commented GPU flags might be important for Chrome stability in headless CI environments. These flags are commonly used to resolve rendering issues in containerized environments.

Consider uncommenting and testing these flags:

    if (process.env.CI) {
-      // args.push("--ignore-gpu-blocklist", "--use-gl=angle", "--use-angle=gl-egl");
+      args.push("--ignore-gpu-blocklist", "--use-gl=angle", "--use-angle=gl-egl");
    }

If these flags cause issues, you might also consider adding --disable-gpu as an alternative for CI environments.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4bc134f and 4c2eacd.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (6)
  • .github/actions/console-web-ui-testing/action.yml (1 hunks)
  • .github/workflows/console-web-docker-build.yml (1 hunks)
  • apps/deploy-web/package.json (1 hunks)
  • apps/deploy-web/playwright.config.ts (2 hunks)
  • apps/deploy-web/tests/deploy-hello-world.spec.ts (1 hunks)
  • apps/deploy-web/tests/fixture/context-with-extension.ts (3 hunks)
🚧 Files skipped from review as they are similar to previous changes (5)
  • apps/deploy-web/tests/deploy-hello-world.spec.ts
  • .github/actions/console-web-ui-testing/action.yml
  • apps/deploy-web/package.json
  • apps/deploy-web/playwright.config.ts
  • .github/workflows/console-web-docker-build.yml
🧰 Additional context used
🪛 Biome (1.9.4)
apps/deploy-web/tests/fixture/context-with-extension.ts

[error] 22-22: Unexpected empty object pattern.

(lint/correctness/noEmptyPattern)

⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: test-deploy-web-build
  • GitHub Check: test-beta
🔇 Additional comments (1)
apps/deploy-web/tests/fixture/context-with-extension.ts (1)

71-71: Good addition of explicit cleanup.

Adding explicit page cleanup is a good practice for resource management and preventing potential memory leaks in test environments.

@stalniy stalniy force-pushed the chore/e2e-tests branch 2 times, most recently from e156c04 to 728ce4f Compare May 29, 2025 05:21
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
apps/deploy-web/tests/fixture/context-with-extension.ts (1)

21-22: Fix the empty object pattern flagged by static analysis.

The empty object pattern {} is flagged by the linter and should be addressed as previously noted.

Consider using one of the suggested approaches from the previous review:

  • Use async (_, use) => if no parameters are needed
  • Use async ({ }, use) => with a space for clarity
🧰 Tools
🪛 Biome (1.9.4)

[error] 22-22: Unexpected empty object pattern.

(lint/correctness/noEmptyPattern)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e156c04 and 728ce4f.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (8)
  • .github/actions/console-web-ui-testing/action.yml (1 hunks)
  • .github/workflows/console-web-docker-build.yml (1 hunks)
  • apps/deploy-web/package.json (1 hunks)
  • apps/deploy-web/playwright.config.ts (2 hunks)
  • apps/deploy-web/tests/deploy-hello-world.spec.ts (1 hunks)
  • apps/deploy-web/tests/fixture/context-with-extension.ts (3 hunks)
  • apps/deploy-web/tests/fixture/wallet-setup.ts (2 hunks)
  • apps/deploy-web/tests/uiState/isWalletConnected.ts (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (6)
  • .github/actions/console-web-ui-testing/action.yml
  • apps/deploy-web/playwright.config.ts
  • apps/deploy-web/package.json
  • apps/deploy-web/tests/uiState/isWalletConnected.ts
  • apps/deploy-web/tests/deploy-hello-world.spec.ts
  • .github/workflows/console-web-docker-build.yml
🧰 Additional context used
🧬 Code Graph Analysis (1)
apps/deploy-web/tests/fixture/wallet-setup.ts (1)
apps/deploy-web/src/utils/apiUtils.ts (1)
  • balance (45-47)
🪛 Biome (1.9.4)
apps/deploy-web/tests/fixture/context-with-extension.ts

[error] 22-22: Unexpected empty object pattern.

(lint/correctness/noEmptyPattern)

⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: test-deploy-web-build
  • GitHub Check: test-beta
🔇 Additional comments (2)
apps/deploy-web/tests/fixture/context-with-extension.ts (1)

67-67: Good cleanup practice!

Adding explicit cleanup with await extPage.close() ensures proper resource management and prevents potential memory leaks in test runs.

apps/deploy-web/tests/fixture/wallet-setup.ts (1)

99-104: Good optimization to avoid unnecessary faucet requests.

The balance check logic is sound and will help reduce load on the faucet service by skipping requests when the wallet already has sufficient funds (10 uakt threshold).

@stalniy stalniy force-pushed the chore/e2e-tests branch 5 times, most recently from 06f0d1e to c096c4b Compare May 29, 2025 06:40
@stalniy stalniy enabled auto-merge (squash) May 29, 2025 06:59
@stalniy stalniy force-pushed the chore/e2e-tests branch from c096c4b to 902621f Compare May 29, 2025 07:08
@stalniy stalniy merged commit 214c40a into main May 29, 2025
29 checks passed
@stalniy stalniy deleted the chore/e2e-tests branch May 29, 2025 08:26
stalniy added a commit that referenced this pull request Nov 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments