diff --git a/qase-python-commons/README.md b/qase-python-commons/README.md index f435756a..e2e1d60f 100644 --- a/qase-python-commons/README.md +++ b/qase-python-commons/README.md @@ -24,6 +24,7 @@ Core library for all Qase Python reporters. Contains the complete configuration - [Single Project (testops)](#single-project-testops) - [Multiple Projects (testops_multi)](#multiple-projects-testops_multi) - [Environment Variables](#environment-variables) +- [Profilers](#profilers) - [Additional Features](#additional-features) - [Status Mapping](#status-mapping) - [Status Filtering](#status-filtering) @@ -267,6 +268,46 @@ export QASE_PYTEST_CAPTURE_LOGS="true" --- +## Profilers + +Profilers automatically track operations during test execution and send them as steps to Qase TestOps. + +| Profiler | Description | Documentation | +|----------|-------------|---------------| +| `network` | Tracks HTTP requests (requests, urllib3) | [Network Profiler](docs/NETWORK_PROFILER.md) | +| `db` | Tracks database operations | [Database Profiler](docs/DATABASE_PROFILERS.md) | +| `sleep` | Tracks sleep calls | — | + +Enable profilers in `qase.config.json`: + +```json +{ + "profilers": ["network", "db"] +} +``` + +Or via environment variable: + +```bash +export QASE_PROFILERS="network,db" +``` + +The `profilers` array supports both string and object formats. Use the object format to configure profiler-specific options: + +```json +{ + "profilers": [ + { + "name": "network", + "excludeHosts": ["telemetry.local", "monitoring.internal"] + }, + "db" + ] +} +``` + +--- + ## Additional Features ### Status Mapping diff --git a/qase-python-commons/changelog.md b/qase-python-commons/changelog.md index c0f923ab..dc0fd41f 100644 --- a/qase-python-commons/changelog.md +++ b/qase-python-commons/changelog.md @@ -1,3 +1,10 @@ +# qase-python-commons@5.0.3 + +## What's new + +- Added support for excluding hosts from the network profiler. You can now specify a list of hosts to exclude using the `excludeHosts` option in the profilers config or the `QASE_PROFILER_NETWORK_EXCLUDE_HOSTS` environment variable. The Qase API host is always excluded automatically. Resolves [#455](https://github.com/qase-tms/qase-python/issues/455). +- Fixed a bug where the `requests` library wrapper in the network profiler did not filter excluded domains (only the `urllib3` wrapper did). + # qase-python-commons@5.0.2 ## What's new diff --git a/qase-python-commons/docs/NETWORK_PROFILER.md b/qase-python-commons/docs/NETWORK_PROFILER.md new file mode 100644 index 00000000..03044151 --- /dev/null +++ b/qase-python-commons/docs/NETWORK_PROFILER.md @@ -0,0 +1,113 @@ +# Network Profiler + +## Overview + +The Network Profiler automatically tracks and logs all HTTP requests during test execution. It captures request method, URL, headers, body, and response status, then sends this data as steps to Qase TestOps for detailed analysis and debugging. + +## Supported Libraries + +The profiler supports the following HTTP libraries: + +- **requests** — Via `requests.Session.send` +- **urllib3** — Via `urllib3.PoolManager.request` + +## Configuration + +### Enable Network Profiler + +Add `"network"` to the `profilers` array in your `qase.config.json`: + +```json +{ + "profilers": ["network"] +} +``` + +Or via environment variable: + +```bash +export QASE_PROFILERS="network" +``` + +### Exclude Hosts + +By default, the Qase API host is always excluded from tracking. You can exclude additional hosts using the `excludeHosts` option or the `QASE_PROFILER_NETWORK_EXCLUDE_HOSTS` environment variable. + +Hosts are matched using substring matching — if any excluded host string is found within the request URL, the request will be skipped. + +#### Config file + +Use the object format in the `profilers` array: + +```json +{ + "profilers": [ + { + "name": "network", + "excludeHosts": ["telemetry.local", "monitoring.internal"] + } + ] +} +``` + +#### Environment variable + +```bash +export QASE_PROFILER_NETWORK_EXCLUDE_HOSTS="telemetry.local,monitoring.internal" +``` + +#### Combining both + +The environment variable and config file values are independent — the environment variable sets the exclude list directly (it does not merge with the config file). The Qase API host (`testops.api.host`) is always excluded automatically regardless of this setting. + +### Multiple Profilers + +You can enable multiple profilers simultaneously, mixing string and object formats: + +```json +{ + "profilers": [ + { + "name": "network", + "excludeHosts": ["telemetry.local"] + }, + "db", + "sleep" + ] +} +``` + +## Collected Data + +For each HTTP request, the profiler collects the following information: + +### Request Information + +- **method** — HTTP method (GET, POST, PUT, DELETE, etc.) +- **url** — Full request URL +- **body** — Request body (when available) +- **headers** — Request headers (when available) + +### Response Information + +- **status_code** — HTTP response status code +- **response_body** — Response body (only for failed requests with status >= 400) +- **response_headers** — Response headers (only for failed requests with status >= 400) + +## Data Format in Qase TestOps + +When sent to Qase TestOps, HTTP requests appear as test steps with: + +- **Step type**: `request` +- **Status**: `passed` for status < 400, `failed` for status >= 400 + +## Automatic Integration + +The network profiler works automatically once enabled in the configuration. No additional code changes are required — it intercepts HTTP operations transparently using monkey patching. + +## Notes + +- The Qase API host is always excluded from tracking to avoid recursive logging +- The profiler only tracks operations that occur after it's enabled +- Failed responses (status >= 400) include response body and headers for debugging +- The profiler handles errors gracefully and won't break your HTTP operations diff --git a/qase-python-commons/pyproject.toml b/qase-python-commons/pyproject.toml index 6b6d4cd4..aec9402f 100644 --- a/qase-python-commons/pyproject.toml +++ b/qase-python-commons/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "qase-python-commons" -version = "5.0.2" +version = "5.0.3" description = "A library for Qase TestOps and Qase Report" readme = "README.md" authors = [{name = "Qase Team", email = "support@qase.io"}] @@ -60,8 +60,7 @@ passenv = commands = pytest --cov-config=pyproject.toml {posargs} extras = - all - testing + testing """ [tool.setuptools.packages.find] diff --git a/qase-python-commons/src/qase/commons/config.py b/qase-python-commons/src/qase/commons/config.py index 5047b7b1..c1f48ad2 100644 --- a/qase-python-commons/src/qase/commons/config.py +++ b/qase-python-commons/src/qase/commons/config.py @@ -254,6 +254,11 @@ def __load_env_config(self): if key == 'QASE_PROFILERS': self.config.set_profilers(value.split(',')) + if key == 'QASE_PROFILER_NETWORK_EXCLUDE_HOSTS': + self.config.network_profiler.set_exclude_hosts( + [h.strip() for h in value.split(',')] + ) + if key == 'QASE_DEBUG': self.config.set_debug(value) diff --git a/qase-python-commons/src/qase/commons/models/config/qaseconfig.py b/qase-python-commons/src/qase/commons/models/config/qaseconfig.py index 47da37d0..732c5e31 100644 --- a/qase-python-commons/src/qase/commons/models/config/qaseconfig.py +++ b/qase-python-commons/src/qase/commons/models/config/qaseconfig.py @@ -40,6 +40,16 @@ def set_path(self, path: str): self.path = path +class NetworkProfilerConfig(BaseModel): + exclude_hosts: List[str] = None + + def __init__(self): + self.exclude_hosts = [] + + def set_exclude_hosts(self, hosts: List[str]): + self.exclude_hosts = hosts + + class QaseConfig(BaseModel): mode: Mode = None fallback: Mode = None @@ -51,6 +61,7 @@ class QaseConfig(BaseModel): testops_multi: TestopsMultiConfig = None report: ReportConfig = None profilers: list = None + network_profiler: NetworkProfilerConfig = None framework: Framework = None exclude_params: list = None status_mapping: Dict[str, str] = None @@ -66,6 +77,7 @@ def __init__(self): self.execution_plan = ExecutionPlan() self.framework = Framework() self.profilers = [] + self.network_profiler = NetworkProfilerConfig() self.exclude_params = [] self.status_mapping = {} self.logging = LoggingConfig() @@ -82,7 +94,16 @@ def set_environment(self, environment: str): self.environment = environment def set_profilers(self, profilers: list): - self.profilers = profilers + self.profilers = [] + for item in profilers: + if isinstance(item, str): + self.profilers.append(item) + elif isinstance(item, dict): + name = item.get("name") + if name: + self.profilers.append(name) + if name == "network" and "excludeHosts" in item: + self.network_profiler.set_exclude_hosts(item["excludeHosts"]) def set_root_suite(self, root_suite: str): self.root_suite = root_suite diff --git a/qase-python-commons/src/qase/commons/profilers/network.py b/qase-python-commons/src/qase/commons/profilers/network.py index ac69b011..073b4627 100644 --- a/qase-python-commons/src/qase/commons/profilers/network.py +++ b/qase-python-commons/src/qase/commons/profilers/network.py @@ -2,6 +2,7 @@ import threading import uuid from functools import wraps +from typing import List from ..models.runtime import Runtime from ..models.step import Step, StepRequestData, StepType @@ -9,13 +10,16 @@ class NetworkProfiler: _instance = None - def __init__(self, runtime: Runtime, skip_domain: str, track_on_fail: bool = True): + def __init__(self, runtime: Runtime, skip_domains: List[str] = None, track_on_fail: bool = True): self._original_functions = {} self.runtime = runtime - self.skip_domain = skip_domain + self.skip_domains = skip_domains or [] self.track_on_fail = track_on_fail self.step = None + def _should_skip(self, url: str) -> bool: + return any(domain in url for domain in self.skip_domains) + def enable(self): if 'requests' in sys.modules: import requests @@ -39,29 +43,31 @@ def disable(self): def _requests_send_wrapper(self, func): @wraps(func) def wrapper(self, request, *args, **kwargs): - NetworkProfilerSingleton.get_instance()._log_request(request) + profiler = NetworkProfilerSingleton.get_instance() + if profiler._should_skip(request.url): + return func(self, request, *args, **kwargs) + + profiler._log_request(request) response = func(self, request, *args, **kwargs) - NetworkProfilerSingleton.get_instance()._log_response(response) + profiler._log_response(response) return response return wrapper def _urllib3_request_wrapper(self, func): - skip_domain = self.skip_domain - @wraps(func) def wrapper(self, method, url, *args, **kwargs): - if skip_domain in url: + profiler = NetworkProfilerSingleton.get_instance() + if profiler._should_skip(url): return func(self, method, url, *args, **kwargs) - interceptor = NetworkProfilerSingleton.get_instance() request = lambda: None request.method = method request.url = url - interceptor._log_request(request) + profiler._log_request(request) response = func(self, method, url, *args, **kwargs) - interceptor._log_response(response, url=url) + profiler._log_response(response, url=url) return response return wrapper diff --git a/qase-python-commons/src/qase/commons/reporters/core.py b/qase-python-commons/src/qase/commons/reporters/core.py index 5243b6e3..65efaea5 100644 --- a/qase-python-commons/src/qase/commons/reporters/core.py +++ b/qase-python-commons/src/qase/commons/reporters/core.py @@ -154,8 +154,10 @@ def setup_profilers(self, runtime: Runtime) -> None: if profiler == "network": # Lazy import from ..profilers import NetworkProfilerSingleton + skip_domains = [self.config.testops.api.host] + skip_domains.extend(self.config.network_profiler.exclude_hosts) NetworkProfilerSingleton.init(runtime=runtime, - skip_domain=self.config.testops.api.host) + skip_domains=skip_domains) self.profilers.append(NetworkProfilerSingleton.get_instance()) if profiler == "sleep": from ..profilers import SleepProfiler diff --git a/qase-python-commons/tests/tests_qase_commons/test_network_profiler.py b/qase-python-commons/tests/tests_qase_commons/test_network_profiler.py new file mode 100644 index 00000000..8955201b --- /dev/null +++ b/qase-python-commons/tests/tests_qase_commons/test_network_profiler.py @@ -0,0 +1,145 @@ +import os +import json +import tempfile +from unittest.mock import patch, MagicMock + +from qase.commons.models.config.qaseconfig import QaseConfig, NetworkProfilerConfig +from qase.commons.models.runtime import Runtime +from qase.commons.profilers.network import NetworkProfiler, NetworkProfilerSingleton + + +class TestNetworkProfilerConfig: + def test_default_initialization(self): + config = NetworkProfilerConfig() + assert config.exclude_hosts == [] + + def test_set_exclude_hosts(self): + config = NetworkProfilerConfig() + config.set_exclude_hosts(["telemetry.local", "monitoring.internal"]) + assert config.exclude_hosts == ["telemetry.local", "monitoring.internal"] + + def test_qase_config_has_network_profiler(self): + config = QaseConfig() + assert isinstance(config.network_profiler, NetworkProfilerConfig) + assert config.network_profiler.exclude_hosts == [] + + +class TestSetProfilersMixedFormat: + def test_string_only(self): + config = QaseConfig() + config.set_profilers(["network", "db"]) + assert config.profilers == ["network", "db"] + assert config.network_profiler.exclude_hosts == [] + + def test_dict_format_with_exclude_hosts(self): + config = QaseConfig() + config.set_profilers([ + {"name": "network", "excludeHosts": ["telemetry.local", "monitoring.internal"]}, + "db" + ]) + assert config.profilers == ["network", "db"] + assert config.network_profiler.exclude_hosts == ["telemetry.local", "monitoring.internal"] + + def test_dict_format_without_exclude_hosts(self): + config = QaseConfig() + config.set_profilers([{"name": "network"}]) + assert config.profilers == ["network"] + assert config.network_profiler.exclude_hosts == [] + + def test_dict_format_non_network(self): + config = QaseConfig() + config.set_profilers([{"name": "db", "excludeHosts": ["some.host"]}]) + assert config.profilers == ["db"] + assert config.network_profiler.exclude_hosts == [] + + def test_empty_profilers(self): + config = QaseConfig() + config.set_profilers([]) + assert config.profilers == [] + + +class TestNetworkProfilerShouldSkip: + def test_should_skip_matching_domain(self): + runtime = MagicMock(spec=Runtime) + profiler = NetworkProfiler(runtime=runtime, skip_domains=["api.qase.io", "telemetry.local"]) + assert profiler._should_skip("https://api.qase.io/v1/runs") is True + assert profiler._should_skip("https://telemetry.local/track") is True + + def test_should_not_skip_non_matching_domain(self): + runtime = MagicMock(spec=Runtime) + profiler = NetworkProfiler(runtime=runtime, skip_domains=["api.qase.io"]) + assert profiler._should_skip("https://example.com/api") is False + + def test_should_skip_empty_domains(self): + runtime = MagicMock(spec=Runtime) + profiler = NetworkProfiler(runtime=runtime, skip_domains=[]) + assert profiler._should_skip("https://example.com") is False + + def test_should_skip_substring_match(self): + runtime = MagicMock(spec=Runtime) + profiler = NetworkProfiler(runtime=runtime, skip_domains=["telemetry"]) + assert profiler._should_skip("https://telemetry.example.com/track") is True + assert profiler._should_skip("https://app.telemetry.io/data") is True + + +class TestNetworkProfilerEnvVar: + def test_env_var_parsing(self): + from qase.commons.config import ConfigManager + + with patch.dict(os.environ, { + 'QASE_PROFILER_NETWORK_EXCLUDE_HOSTS': 'telemetry.local,monitoring.internal' + }): + config_manager = ConfigManager() + assert config_manager.config.network_profiler.exclude_hosts == [ + "telemetry.local", "monitoring.internal" + ] + + def test_env_var_single_host(self): + from qase.commons.config import ConfigManager + + with patch.dict(os.environ, { + 'QASE_PROFILER_NETWORK_EXCLUDE_HOSTS': 'telemetry.local' + }): + config_manager = ConfigManager() + assert config_manager.config.network_profiler.exclude_hosts == ["telemetry.local"] + + +class TestNetworkProfilerFileConfig: + def test_mixed_format_from_file(self): + from qase.commons.config import ConfigManager + + config_data = { + "profilers": [ + {"name": "network", "excludeHosts": ["telemetry.local"]}, + "db" + ] + } + + with tempfile.NamedTemporaryFile(mode='w', suffix='.json', delete=False) as f: + json.dump(config_data, f) + config_file = f.name + + try: + config_manager = ConfigManager(config_file) + assert config_manager.config.profilers == ["network", "db"] + assert config_manager.config.network_profiler.exclude_hosts == ["telemetry.local"] + finally: + os.unlink(config_file) + + def test_string_format_backward_compatible(self): + from qase.commons.config import ConfigManager + + config_data = { + "profilers": ["network", "db"] + } + + with tempfile.NamedTemporaryFile(mode='w', suffix='.json', delete=False) as f: + json.dump(config_data, f) + config_file = f.name + + try: + config_manager = ConfigManager(config_file) + assert config_manager.config.profilers == ["network", "db"] + assert config_manager.config.network_profiler.exclude_hosts == [] + finally: + os.unlink(config_file)