Skip to content

chore: refactor functional api tests to rely on fs lock#2567

Merged
stalniy merged 2 commits intomainfrom
chore/api-functional-tests
Jan 26, 2026
Merged

chore: refactor functional api tests to rely on fs lock#2567
stalniy merged 2 commits intomainfrom
chore/api-functional-tests

Conversation

@stalniy
Copy link
Contributor

@stalniy stalniy commented Jan 26, 2026

Why

We currently pre-topup test wallets in Jest global setup / custom environment. While this avoids faucet contention, it significantly hurts developer experience:

  • setup logic creates a new wallet per spec file, and tops up it. Very often we don't need this (only 10 test files require wallet topup and 22 doesn't require it)
  • functional tests startup is very long, doesn't allow to do tests filtering until it finishes topup logic. This makes feedback loops slow and frustrating during local development. Every change in files -> waiting for wallets topup

What

This PR replaces eager, global topups with a lazy funding model:

  • wallets are funded only when a test actually needs it
  • faucet access is safely serialized across Jest workers via fs lock
  • transient faucet failures and account sequence mismatches are retried

This approach does not aim to make faucet-dependent tests faster, but it greatly improves DX and test isolation by removing unnecessary global side effects and allowing developers to quickly run or filter tests without paying the faucet cost upfront. However it potentially can speed up functional tests execution because we don't need to wait for redundant wallet creation and topups.

Summary by CodeRabbit

  • Tests
    • Added pre-test wallet top-ups across functional suites and a beforeAll funding step for more reliable tests.
    • Simplified and concurrentized test wallet initialization with memoized mnemonic retrieval for faster setup.
  • Reliability
    • Introduced cross-process locking and centralized retry policies to reduce transient failures.
    • Added persistent mnemonic caching and startup cleanup to prevent stale-lock and wallet flakiness.
  • New Features
    • Added a faucet-backed wallet top-up utility to ensure minimum balances during tests.

✏️ Tip: You can customize this high-level summary in your review settings.

@stalniy stalniy requested a review from a team as a code owner January 26, 2026 13:06
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 26, 2026

📝 Walkthrough

Walkthrough

Adds a filesystem-based cross-process lock and retry policies, a faucet-backed topUpWallet utility used by many functional tests, centralizes mnemonic setup in the custom Jest environment, simplifies TestWalletService to memoized persistent mnemonic storage, and wires topUpWallet into multiple test suites' beforeAll hooks.

Changes

Cohort / File(s) Summary
Locking & Retry
apps/api/test/services/fs-lock.ts, apps/api/test/services/retry-policies.ts
New fs-lock implementation with staleness detection, timeouts, jittered retries, and convenience faucet lock helpers; centralized retry policies and detectors for faucet and account sequence errors.
Wallet Top-up Utility
apps/api/test/services/topUpWallet.ts
New topUpWallet(...) and TopUpWalletOptions: checks balance, acquires faucet lock, posts to faucet with retry policy, and waits for balance to reach minimum.
Jest env & global setup
apps/api/test/custom-jest-environment.ts, apps/api/test/setup-global-functional.ts
Consolidates per-wallet mnemonic assignment into a private #setOrGenerateWallet helper; global setup now force-unlocks stale faucet locks on startup.
TestWalletService simplification
apps/api/test/services/test-wallet.service.ts
Simplified service: replaces complex faucet/balance orchestration with memoized persistent mnemonic cache and new getStoredMnemonic(path) method.
Functional tests wiring
apps/api/test/functional/*.spec.ts
(e.g., api-key.spec.ts, balances.spec.ts, bids.spec.ts, create-deployment.spec.ts, deployment-setting.spec.ts, managed-api-deployment-flow.spec.ts, sign-and-broadcast-tx.spec.ts, start-trial.spec.ts, stripe-webhook.spec.ts, wallets-refill.spec.ts)
Adds import and beforeAll(...) calling topUpWallet() (some with custom minAmount) to ensure funding before tests.
New fs-lock usage helpers
apps/api/test/setup-global-functional.ts, apps/api/test/services/topUpWallet.ts
Introduces and uses forceUnlockFaucet / withFaucetLock to coordinate faucet operations across processes.

Sequence Diagram(s)

sequenceDiagram
    actor Test as Functional Test
    participant WalletUtil as topUpWallet()
    participant FsLock as FS Lock File
    participant BalanceSvc as Node Balance
    participant FaucetSvc as Faucet Service
    participant Waiter as Balance Wait Policy

    Test->>WalletUtil: Call topUpWallet(options)
    WalletUtil->>BalanceSvc: Get current balance
    BalanceSvc-->>WalletUtil: Current balance

    alt balance < minAmount
        WalletUtil->>FsLock: Acquire faucet lock (withFaucetLock)
        FsLock-->>WalletUtil: Lock acquired

        WalletUtil->>BalanceSvc: Re-check balance
        BalanceSvc-->>WalletUtil: Current balance (still low)

        WalletUtil->>FaucetSvc: POST top-up (faucetRetryPolicy)
        FaucetSvc-->>WalletUtil: 2xx or retry/errors

        WalletUtil->>Waiter: Poll balance (balanceWaitPolicy)
        Waiter->>BalanceSvc: Repeated checks
        BalanceSvc-->>Waiter: Updated balance
        Waiter-->>WalletUtil: Balance >= minAmount

        WalletUtil->>FsLock: Release faucet lock
        FsLock-->>WalletUtil: Lock removed
    else balance >= minAmount
        WalletUtil-->>Test: Return current balance (no-op)
    end

    WalletUtil-->>Test: Return final balance
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested reviewers

  • baktun14
  • ygrishajev

"I dug a cozy burrow, left a lock to keep things bright,
I nudged the faucet gently until each balance right,
Retries danced like moonbeams, waiting patient, snug and small,
Now tests can hop like springtime — steady, safe, and tall." 🐇✨

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: refactoring functional API tests to use filesystem-based locking for wallet top-ups instead of eager global setup.
Docstring Coverage ✅ Passed Docstring coverage is 82.35% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Fix all issues with AI agents
In `@apps/api/test/services/fs-lock.ts`:
- Around line 26-155: The current logic in withFsLock may delete a live lock if
staleLockThreshold is shorter than the maximum expected critical-section
duration; validate and enforce a safe threshold at start of withFsLock by
checking the merged options (DEFAULT_OPTIONS + opts) and either: 1) throw a
clear Error if options.staleLockThreshold < options.timeout, or 2) automatically
set options.staleLockThreshold = Math.max(options.staleLockThreshold,
options.timeout) (pick one approach consistently), and document this behavior;
update references to FsLockOptions, DEFAULT_OPTIONS, withFsLock, isLockStale,
and tryDeleteStaleLock so stale checks/resets use the validated/adjusted value
and consider adding a comment suggesting a heartbeat alternative for
long-running critical sections.

In `@apps/api/test/services/test-wallet.service.ts`:
- Around line 28-35: getStoredMnemonic is race-prone because it does a
check-then-await on generateMnemonic; memoize the in-flight promise so
concurrent callers share the same generation and avoid duplicate saves.
Implement a companion map (e.g., mnemonicPromises) keyed by fileName; in
getStoredMnemonic, if mnemonics[fileName] exists return it, else if
mnemonicPromises[fileName] exists await and return its resolved value, otherwise
set mnemonicPromises[fileName] = generateMnemonic(), await it, assign the
resolved string to mnemonics[fileName], delete mnemonicPromises[fileName], call
saveCache() once, and return the mnemonic; use the existing function names
getStoredMnemonic, generateMnemonic, mnemonics, and saveCache to locate and
update the code.
🧹 Nitpick comments (2)
apps/api/test/services/topUpWallet.ts (2)

11-11: Consider handling missing config gracefully.

The config variable uses non-null assertion (config!) at lines 21 and 46. If the .env.functional.test file is missing or malformed, this will cause a runtime error with an unhelpful message.

Optional: Add validation
 const { parsed: config } = dotenvExpand.expand(dotenv.config({ path: "env/.env.functional.test" }));
+
+if (!config) {
+  throw new Error("Failed to load env/.env.functional.test - ensure the file exists");
+}

This would provide a clearer error message if the config file is missing.


81-84: Consider using response.ok for cleaner status check.

The current logic is correct but response.ok is more idiomatic for checking successful HTTP responses.

Suggested simplification
-  if (response.status >= 300 || response.status < 200) {
+  if (!response.ok) {
     const errorBody = await response.text().catch(() => "Unknown error");
     throw new Error(`Faucet request failed with status ${response.status}: ${errorBody}`);
   }

@codecov
Copy link

codecov bot commented Jan 26, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 50.64%. Comparing base (626e139) to head (34ebb57).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

❌ Your project status has failed because the head coverage (79.55%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2567      +/-   ##
==========================================
- Coverage   50.94%   50.64%   -0.30%     
==========================================
  Files        1069     1059      -10     
  Lines       29815    29466     -349     
  Branches     6598     6540      -58     
==========================================
- Hits        15188    14923     -265     
+ Misses      14344    14147     -197     
- Partials      283      396     +113     
Flag Coverage Δ *Carryforward flag
api 79.55% <ø> (-0.03%) ⬇️
deploy-web 31.45% <ø> (ø) Carriedforward from 626e139
log-collector ?
notifications 87.94% <ø> (ø) Carriedforward from 626e139
provider-console 81.48% <ø> (ø) Carriedforward from 626e139
provider-proxy 84.35% <ø> (ø) Carriedforward from 626e139

*This pull request uses carry forward flags. Click here to find out more.
see 49 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@stalniy stalniy force-pushed the chore/api-functional-tests branch from 900609d to 1843628 Compare January 26, 2026 13:41
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@apps/api/test/services/topUpWallet.ts`:
- Around line 11-22: The code uses a non-null assertion on config when
constructing BalanceHttpService (const { parsed: config } =
dotenvExpand.expand(dotenv.config(...)); new BalanceHttpService({ baseURL:
config!REST_API_NODE_URL })), which can throw cryptic runtime errors if
env/.env.functional.test is missing or malformed; replace the assertion with an
explicit check: after calling dotenvExpand.expand(dotenv.config(...)) validate
that parsed (config) is defined and that config.REST_API_NODE_URL exists, and if
not throw a clear Error (or call process.exit with a descriptive message) before
creating BalanceHttpService so BalanceHttpService always receives a valid
baseURL.
🧹 Nitpick comments (2)
apps/api/test/services/test-wallet.service.ts (1)

35-37: Minor: Redundant nullish coalescing assignment.

The ??= on line 36 is unnecessary since we're already inside the if (!this.mnemonics[fileName]) block. Consider simplifying:

    if (!this.mnemonics[fileName]) {
-     this.mnemonics[fileName] ??= await this.generateMnemonic();
+     this.mnemonics[fileName] = await this.generateMnemonic();
      this.saveCache();
    }
apps/api/test/services/fs-lock.ts (1)

189-189: Consider using path.join for cross-platform consistency.

FAUCET_LOCK_PATH uses template literal string concatenation. For better cross-platform compatibility, consider using path.join:

-export const FAUCET_LOCK_PATH = `${os.tmpdir()}/cosmos-faucet.lock`;
+export const FAUCET_LOCK_PATH = path.join(os.tmpdir(), "cosmos-faucet.lock");

baktun14
baktun14 previously approved these changes Jan 26, 2026
@stalniy stalniy force-pushed the chore/api-functional-tests branch from 1843628 to 34ebb57 Compare January 26, 2026 16:55
@stalniy stalniy merged commit 03f7d31 into main Jan 26, 2026
66 of 67 checks passed
@stalniy stalniy deleted the chore/api-functional-tests branch January 26, 2026 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments