feat(telemetry): Create a new SCARF telemetry event to track the registered email address by mpawlow · Pull Request #10432 · langflow-ai/langflow

mpawlow · 2025-10-28T21:02:44Z

Overview

✅ Created a new telemetry schema for the registered email address
✅ Send email telemetry events on the following triggers
- (1) Every time a new email address is registered via the new POST registration API endpoint
- (2) Once at Langflow Desktop start-up in the Telemetry Service start-up lifecycle method
✅ Implemented 4 new test scenarios for testing the new Email payload schema
✅ Implement 6 new test scenarios for testing the "get email model" function from the registered email utility that invokes the registration API endpoint and caches the result

Related PRs

New Test Scenario Results - Email Payload Schema

src/backend/tests/unit/services/telemetry/test_telemetry_schema.py::TestEmailPayload.test_email_payload_initialization ✓
src/backend/tests/unit/services/telemetry/test_telemetry_schema.py::TestEmailPayload.test_email_payload_initialization_without_client_type ✓
src/backend/tests/unit/services/telemetry/test_telemetry_schema.py::TestEmailPayload.test_email_payload_initialization_with_invalid_email ✓
src/backend/tests/unit/services/telemetry/test_telemetry_schema.py::TestEmailPayload.test_email_payload_serialization ✓

New Test Scenario Results - Get Email Model

 src/backend/tests/unit/utils/test_registered_email_util.py::TestGetEmailModel.test_get_email_model_success ✓
 src/backend/tests/unit/utils/test_registered_email_util.py::TestGetEmailModel.test_get_email_model_oserror ✓
 src/backend/tests/unit/utils/test_registered_email_util.py::TestGetEmailModel.test_get_email_model_cached ✓ 
 src/backend/tests/unit/utils/test_registered_email_util.py::TestGetEmailModel.test_get_email_model_invalid_registration ✓
 src/backend/tests/unit/utils/test_registered_email_util.py::TestGetEmailModel.test_get_email_model_invalid_email_missing ✓
 src/backend/tests/unit/utils/test_registered_email_util.py::TestGetEmailModel.test_get_email_model_invalid_email_format ✓

Summary by CodeRabbit

Chores
- Enhanced telemetry data collection to include registered email information when available.
- Optimized platform detection with caching for improved performance.

coderabbitai · 2025-10-28T21:03:16Z

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

The changes add email registration support to telemetry collection by introducing a new utility function to retrieve registered email addresses and integrating it into the telemetry service. The service also caches the platform computation to avoid repeated runtime calls.

Changes

Cohort / File(s)	Summary
Email Registration Telemetry `src/backend/base/langflow/utils/registered_email_util.py`, `src/backend/base/langflow/services/telemetry/service.py`	New utility module exports `get_registered_email_address()` with comprehensive error handling for loading and validating registered email addresses. Telemetry service imports and uses this function to augment common telemetry fields with email when available. Also introduces cached `is_langflow_desktop` flag to compute platform value once instead of at runtime.

Sequence Diagram(s)

sequenceDiagram
    participant TelemetryService
    participant RegisteredEmailUtil
    participant LoadRegistrations
    
    TelemetryService->>TelemetryService: Initialize (check is_langflow_desktop cached)
    TelemetryService->>TelemetryService: Build common_fields
    TelemetryService->>RegisteredEmailUtil: get_registered_email_address()
    RegisteredEmailUtil->>LoadRegistrations: load_registrations()
    LoadRegistrations-->>RegisteredEmailUtil: registration_data
    RegisteredEmailUtil->>RegisteredEmailUtil: Validate & extract email
    alt Email valid
        RegisteredEmailUtil-->>TelemetryService: email (str)
    else Email invalid or error
        RegisteredEmailUtil-->>TelemetryService: "" (empty string)
    end
    TelemetryService->>TelemetryService: Add email to common_fields (if present)

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

New utility file is self-contained with straightforward validation logic and comprehensive error handling
Telemetry service modifications are localized to initialization and field augmentation
Integration is minimal and follows established patterns
Consider verifying error handling scenarios and logging consistency in the utility function

Pre-merge checks and finishing touches

❌ Failed checks (1 error, 4 warnings, 1 inconclusive)

Check name	Status	Explanation	Resolution
Test Coverage For New Implementations	❌ Error	This PR introduces two significant code changes: (1) a new utility file `src/backend/base/langflow/utils/registered_email_util.py` with the function `get_registered_email_address()`, and (2) modifications to `src/backend/base/langflow/services/telemetry/service.py` to integrate this utility and augment telemetry events with the registered email. However, the PR contains no new or updated test files. No tests were added for the new `get_registered_email_address()` utility function, and no existing telemetry tests were modified to verify the new email augmentation behavior. This leaves the new functionality untested and creates potential blind spots for regressions.	Add appropriate test coverage by creating a new unit test file `src/backend/tests/unit/utils/test_registered_email_util.py` to test the `get_registered_email_address()` function with various scenarios (valid email, invalid registrations, error handling, and edge cases). Additionally, modify existing telemetry tests (such as `test_telemetry.py`) to verify that the email is correctly augmented in common telemetry fields when available and only for the desktop platform as intended by the design.
Docstring Coverage	⚠️ Warning	Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%.	You can run `@coderabbitai generate docstrings` to improve docstring coverage.
Test Quality And Coverage	⚠️ Warning	The PR introduces a new utility function `get_registered_email_address()` with 45 lines of code containing multiple error paths and a complex integration into TelemetryService, but fails to include any corresponding tests. The new code has at least 6 distinct error scenarios, 3 exception types, and multiple validation branches that should be tested. The project establishes clear patterns of comprehensive utility testing (22 existing utility test files with parametrized tests, mocking, and edge case coverage), yet this PR adds a new utility function without a single test, leaving the error handling paths and TelemetryService integration completely untested.
Test File Naming And Structure	⚠️ Warning	The PR introduces a new utility function `get_registered_email_address()` in `src/backend/base/langflow/utils/registered_email_util.py` and modifies the telemetry service to use it, but fails to provide any test coverage. No new test file `test_registered_email_util.py` was created in the existing `src/backend/tests/unit/utils/` directory where similar utility tests are located (e.g., `test_component_utils.py`, `test_compression.py`). Furthermore, existing telemetry tests in `test_telemetry.py` and `test_exception_telemetry.py` were not updated to verify the new email augmentation functionality or the behavior when email is/is not available. The new utility function contains multiple error handling paths and edge cases (invalid registration types, missing email fields, JSON errors, permission errors, OS errors) that lack test coverage for both positive and negative scenarios.	Create a test file `src/backend/tests/unit/utils/test_registered_email_util.py` following the project's existing pattern with test cases covering: successful email retrieval, empty registration list, invalid registration types, missing email field, invalid email strings, and all exception scenarios (JSONDecodeError, PermissionError, OSError). Additionally, update existing telemetry service tests to verify that the email field is correctly added to `common_telemetry_fields` when available, and that it is not added when the function returns an empty string, including tests for the desktop-only context guard.
Title check	⚠️ Warning	The PR title mentions creating a 'new SCARF telemetry event' but the actual changes augment existing events with email data and introduce utility functions—not creating a new event.	Update the title to reflect the actual change: something like 'feat(telemetry): Augment SCARF events with registered email address' or similar to accurately describe augmenting existing events rather than creating a new one.
Excessive Mock Usage Warning	❓ Inconclusive	The PR modifies only two source files without including any test files: it modifies `src/backend/base/langflow/services/telemetry/service.py` and adds `src/backend/base/langflow/utils/registered_email_util.py`. The custom check requires reviewing test files for excessive mock usage, but no test files are present in this PR. While the existing telemetry test suite demonstrates appropriate mock usage patterns (mocking only external dependencies like HTTP clients and async operations), the new code introduced in this PR remains untested and therefore cannot be evaluated under this check. The absence of tests for the new `get_registered_email_address()` function and the TelemetryService email integration means there are no test patterns to assess.	Test files must be added to this PR to enable assessment of this check. The recommended tests should follow the established patterns in the codebase and use mocks judiciously: (1) Mock `load_registrations()` in tests of `get_registered_email_address()` since it's an external dependency, while testing the utility's error handling and validation logic; (2) Mock the external HTTP client in TelemetryService tests but test the actual email field integration with real objects; (3) Use integration tests to verify the email field is correctly populated in `common_telemetry_fields` when the service initializes; (4) Test the desktop-only guard condition to ensure email is only fetched in appropriate contexts. This approach maintains the quality seen in the existing telemetry tests while ensuring comprehensive coverage of the PR's new functionality.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 3

🧹 Nitpick comments (2)

src/backend/base/langflow/utils/registered_email_util.py (2)
37-45: Consolidate repetitive exception handlers.

The three exception handlers (JSONDecodeError, PermissionError, OSError) all perform the same action: log an error and return an empty string. Consider consolidating them into a single handler for cleaner code.

Apply this diff to consolidate the exception handling:
-    except json.JSONDecodeError as e:
-        logger.error(f"Unable to load email registrations: {e}.")
-        return ""
-    except PermissionError as e:
-        logger.error(f"Unable to load email registrations: {e}.")
-        return ""
-    except OSError as e:
+    except (json.JSONDecodeError, PermissionError, OSError) as e:
         logger.error(f"Unable to load email registrations: {e}.")
         return ""
8-45: Consider adding email format validation.

While the function validates that the email is a non-empty string, it doesn't verify that it's a valid email format. Consider adding basic email format validation (e.g., contains '@' and domain) to catch obviously malformed email addresses before they're sent with telemetry data.

You could add a simple validation check after line 28:
# Retrieve email address
email = email_registration.get("email")

# Verify email address is a valid non-zero length string
if not isinstance(email, str) or (len(email) == 0):
    logger.error("Email registration is not a valid non-zero length string.")
    return ""

# Basic email format validation
if "@" not in email or "." not in email.split("@")[-1]:
    logger.error("Email registration is not a valid email format.")
    return ""

return email  # noqa: TRY300

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a703f7b and 0cdfed0.

📒 Files selected for processing (2)

src/backend/base/langflow/services/telemetry/service.py (2 hunks)
src/backend/base/langflow/utils/registered_email_util.py (1 hunks)

🧰 Additional context used

📓 Path-based instructions (1)

{src/backend/**/*.py,tests/**/*.py,Makefile}

📄 CodeRabbit inference engine (.cursor/rules/backend_development.mdc)

{src/backend/**/*.py,tests/**/*.py,Makefile}: Run make format_backend to format Python code before linting or committing changes
Run make lint to perform linting checks on backend Python code

Files:

src/backend/base/langflow/utils/registered_email_util.py
src/backend/base/langflow/services/telemetry/service.py

🧠 Learnings (1)

📚 Learning: 2025-10-29T03:59:53.796Z

Learnt from: ricofurtado
PR: langflow-ai/langflow#10430
File: src/backend/base/langflow/api/v2/registration.py:0-0
Timestamp: 2025-10-29T03:59:53.796Z
Learning: The user registration functionality in `src/backend/base/langflow/api/v2/registration.py` is temporary and will be deprecated when single-user registration is confirmed as a requirement for the langflow project.

Applied to files:

src/backend/base/langflow/utils/registered_email_util.py

🧬 Code graph analysis (1)

src/backend/base/langflow/services/telemetry/service.py (2)

src/backend/base/langflow/utils/registered_email_util.py (1)

get_registered_email_address (8-45)

src/backend/base/langflow/utils/version.py (1)

get_version_info (90-91)

🪛 GitHub Actions: autofix.ci

src/backend/base/langflow/utils/registered_email_util.py

[error] 5-5: ModuleNotFoundError: No module named 'langflow.api.v2.registration' while importing load_registrations in registered_email_util.py during step: 'uv run python scripts/ci/update_starter_projects.py'.

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

GitHub Check: Optimize new Python code in this PR

🔇 Additional comments (2)

src/backend/base/langflow/services/telemetry/service.py (2)
56-62: Good optimization: caching the platform computation.

Computing is_langflow_desktop once and reusing it for the platform field is a good practice that avoids repeated environment variable lookups and string comparisons.

25-25: Fix the broken import chain: langflow.api.v2.registration module does not exist.

The import on line 25 will fail because registered_email_util.py tries to import load_registrations from langflow.api.v2.registration, but this module does not exist anywhere in the codebase. The api/v2 directory only contains files.py and mcp.py.

Either:

Implement the missing langflow.api.v2.registration module with the load_registrations function, or

Correct the import path to the actual location of load_registrations, or

Remove registered_email_util.py and its usages if this functionality is not needed

This is a blocking issue that prevents service.py from being imported at runtime.
⛔ Skipped due to learnings
Learnt from: ricofurtado
PR: langflow-ai/langflow#10430
File: src/backend/base/langflow/api/v2/registration.py:0-0
Timestamp: 2025-10-29T03:59:53.796Z
Learning: The user registration functionality in `src/backend/base/langflow/api/v2/registration.py` is temporary and will be deprecated when single-user registration is confirmed as a requirement for the langflow project.
Learnt from: ricofurtado
PR: langflow-ai/langflow#10430
File: src/backend/base/langflow/api/v2/registration.py:0-0
Timestamp: 2025-10-29T03:55:50.216Z
Learning: In the langflow project, user registration endpoints in API v2 (`src/backend/base/langflow/api/v2/registration.py`) are intentionally designed to be unauthenticated to support Desktop application initialization and onboarding flows.

codecov · 2025-10-30T19:53:49Z

Codecov Report

❌ Patch coverage is 74.74747% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 31.70%. Comparing base (b535950) to head (7c82d4d).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
...ackend/base/langflow/services/telemetry/service.py	38.46%	16 Missing ⚠️
src/backend/base/langflow/api/v2/registration.py	66.66%	6 Missing ⚠️
...ckend/base/langflow/utils/registered_email_util.py	94.23%	3 Missing ⚠️

❌ Your project status has failed because the head coverage (39.77%) is below the target coverage (60.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10432      +/-   ##
==========================================
+ Coverage   31.45%   31.70%   +0.25%     
==========================================
  Files        1328     1330       +2     
  Lines       60153    60460     +307     
  Branches     8994     9059      +65     
==========================================
+ Hits        18920    19171     +251     
- Misses      40326    40376      +50     
- Partials      907      913       +6

Flag	Coverage Δ
backend	`51.26% <74.74%> (+0.13%)`	⬆️
lfx	`39.77% <ø> (+0.48%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...backend/base/langflow/services/telemetry/schema.py	`100.00% <100.00%> (ø)`
...ckend/base/langflow/utils/registered_email_util.py	`94.23% <94.23%> (ø)`
src/backend/base/langflow/api/v2/registration.py	`91.35% <66.66%> (-7.06%)`	⬇️
...ackend/base/langflow/services/telemetry/service.py	`79.74% <38.46%> (-6.49%)`	⬇️

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2025-10-31T04:24:30Z

Frontend Unit Test Coverage Report

Coverage Summary

Lines	Statements	Branches	Functions
	14.66% (3955/26976)	7.45% (1533/20560)	8.99% (532/5913)

Unit Test Results

Tests	Skipped	Failures	Errors	Time
1588	0 💤	0 ❌	0 🔥	19.025s ⏱️

…ess #10432 - Create a new telemetry schema for the registered email address - Implement utility methods to send new telemetry event for the registered email address - Send new telemetry event for the registered email address on telemetry service start-up lifecycle method - Code clean-up, commenting & refactoring

mpawlow · 2025-11-11T04:30:59Z

Status Update

✅ This PR is unblocked
- Dependent PR has been merged into main: feat: Add user registration endpoints in API v2. #10430
- This PR is now based off of the latest from main

mpawlow · 2025-11-11T04:41:26Z

Status Update

❌ PR is blocked
- Nightly build failed again
- See: https://github.com/langflow-ai/langflow/actions/runs/19254877438/job/55047962647?pr=10432

Copilot

Pull Request Overview

This PR introduces email telemetry tracking for Langflow by creating a new EmailPayload schema and implementing collection mechanisms. The feature captures registered email addresses and sends them as telemetry events both during desktop application startup and when users register via the API.

Key Changes:

Added EmailPayload telemetry schema with email validation
Implemented email telemetry sending on registration and desktop startup
Created utility module for retrieving and caching registered email addresses

Reviewed Changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`src/backend/base/langflow/services/telemetry/schema.py`	Added new `EmailPayload` class with email validation using `EmailStr`
`src/backend/base/langflow/services/telemetry/service.py`	Added email telemetry task and `_send_email_telemetry()` method for desktop startup
`src/backend/base/langflow/api/v2/registration.py`	Integrated email telemetry into registration endpoint via `_send_email_telemetry()`
`src/backend/base/langflow/utils/registered_email_util.py`	Implemented email retrieval utility with caching via `_RegisteredEmailCache`
`src/backend/tests/unit/services/telemetry/test_telemetry_schema.py`	Added 4 test cases for `EmailPayload` and updated existing tests with new required fields
`src/backend/tests/unit/utils/test_registered_email_util.py`	Added 6 test cases covering email model retrieval scenarios

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

ogabrielluiz

LGTM

…stered email address - Temporarily add hardcoded registered email address to common SCARF telemetry events - Only send registered email address if it's defined and the context is Langflow Desktop - Implement a utility to load and fetch the registered email address - Bootstrap common telemetry fields with the registered email address (only in the Langflow Desktop context) - Lazy load registered email address - Cache loaded registered email address - Adopt latest email registration storage format / schema - Create a new telemetry schema for the registered email address - Implement utility methods to send new telemetry event for the registered email address - Send new telemetry event for the registered email address on telemetry service start-up lifecycle method - Code clean-up, commenting & refactoring - Implement 4 new test scenarios for testing the new Email payload schema - Fix all pre-existing style errors - Implement 6 new test scenarios for testing the "get email model" function from the registered email utility that invokes the registration API endpoint and caches the result

…stered email address - Address PR feedback from Co-pilot

mpawlow · 2025-11-12T00:28:03Z

Status Update

✅ Added a commit to address the GitHub Co-pilot feedback

mpawlow self-assigned this Oct 28, 2025

mpawlow force-pushed the mp_LFOSS-2632_telemetry branch 2 times, most recently from 673966c to 0cdfed0 Compare October 30, 2025 19:33

mpawlow marked this pull request as ready for review October 30, 2025 19:33

coderabbitai Bot reviewed Oct 30, 2025

View reviewed changes

Comment thread src/backend/base/langflow/services/telemetry/service.py Outdated

Comment thread src/backend/base/langflow/utils/registered_email_util.py Outdated

Comment thread src/backend/base/langflow/utils/registered_email_util.py Outdated

mpawlow force-pushed the mp_LFOSS-2632_telemetry branch from 0cdfed0 to ac74abb Compare October 30, 2025 19:52

mpawlow mentioned this pull request Oct 30, 2025

feat: Add user registration endpoints in API v2. #10430

Merged

mpawlow force-pushed the mp_LFOSS-2632_telemetry branch 2 times, most recently from 7dc178c to 3c06ff7 Compare October 31, 2025 04:19

mpawlow requested a review from ogabrielluiz October 31, 2025 05:16

mpawlow force-pushed the mp_LFOSS-2632_telemetry branch from d878cde to 7245e2f Compare October 31, 2025 21:21

mpawlow changed the title ~~Augment all existing SCARF telemetry events to include the registered…~~ Create a new SCARF telemetry event to track the registered email address Nov 10, 2025

mpawlow force-pushed the mp_LFOSS-2632_telemetry branch from 7245e2f to 947898f Compare November 10, 2025 17:31

mpawlow force-pushed the mp_LFOSS-2632_telemetry branch 2 times, most recently from 22af4f4 to 7bb6cc5 Compare November 11, 2025 02:50

mpawlow changed the title ~~Create a new SCARF telemetry event to track the registered email address~~ feat(telemetry): Create a new SCARF telemetry event to track the registered email address Nov 11, 2025

github-actions Bot added the enhancement New feature or request label Nov 11, 2025

codeflash-ai Bot mentioned this pull request Nov 11, 2025

⚡️ Speed up function get_email_model by 106% in PR #10432 (mp_LFOSS-2632_telemetry) #10556

Closed

mpawlow force-pushed the mp_LFOSS-2632_telemetry branch from 5e11fcc to db9a198 Compare November 11, 2025 03:17