Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions qase-python-commons/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Core library for all Qase Python reporters. Contains the complete configuration
- [Single Project (testops)](#single-project-testops)
- [Multiple Projects (testops_multi)](#multiple-projects-testops_multi)
- [Environment Variables](#environment-variables)
- [Profilers](#profilers)
- [Additional Features](#additional-features)
- [Status Mapping](#status-mapping)
- [Status Filtering](#status-filtering)
Expand Down Expand Up @@ -267,6 +268,46 @@ export QASE_PYTEST_CAPTURE_LOGS="true"

---

## Profilers

Profilers automatically track operations during test execution and send them as steps to Qase TestOps.

| Profiler | Description | Documentation |
|----------|-------------|---------------|
| `network` | Tracks HTTP requests (requests, urllib3) | [Network Profiler](docs/NETWORK_PROFILER.md) |
| `db` | Tracks database operations | [Database Profiler](docs/DATABASE_PROFILERS.md) |
| `sleep` | Tracks sleep calls | — |

Enable profilers in `qase.config.json`:

```json
{
"profilers": ["network", "db"]
}
```

Or via environment variable:

```bash
export QASE_PROFILERS="network,db"
```

The `profilers` array supports both string and object formats. Use the object format to configure profiler-specific options:

```json
{
"profilers": [
{
"name": "network",
"excludeHosts": ["telemetry.local", "monitoring.internal"]
},
"db"
]
}
```

---

## Additional Features

### Status Mapping
Expand Down
7 changes: 7 additions & 0 deletions qase-python-commons/changelog.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
# qase-python-commons@5.0.3

## What's new

- Added support for excluding hosts from the network profiler. You can now specify a list of hosts to exclude using the `excludeHosts` option in the profilers config or the `QASE_PROFILER_NETWORK_EXCLUDE_HOSTS` environment variable. The Qase API host is always excluded automatically. Resolves [#455](https://github.com/qase-tms/qase-python/issues/455).
- Fixed a bug where the `requests` library wrapper in the network profiler did not filter excluded domains (only the `urllib3` wrapper did).

# qase-python-commons@5.0.2

## What's new
Expand Down
113 changes: 113 additions & 0 deletions qase-python-commons/docs/NETWORK_PROFILER.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Network Profiler

## Overview

The Network Profiler automatically tracks and logs all HTTP requests during test execution. It captures request method, URL, headers, body, and response status, then sends this data as steps to Qase TestOps for detailed analysis and debugging.

## Supported Libraries

The profiler supports the following HTTP libraries:

- **requests** — Via `requests.Session.send`
- **urllib3** — Via `urllib3.PoolManager.request`

## Configuration

### Enable Network Profiler

Add `"network"` to the `profilers` array in your `qase.config.json`:

```json
{
"profilers": ["network"]
}
```

Or via environment variable:

```bash
export QASE_PROFILERS="network"
```

### Exclude Hosts

By default, the Qase API host is always excluded from tracking. You can exclude additional hosts using the `excludeHosts` option or the `QASE_PROFILER_NETWORK_EXCLUDE_HOSTS` environment variable.

Hosts are matched using substring matching — if any excluded host string is found within the request URL, the request will be skipped.

#### Config file

Use the object format in the `profilers` array:

```json
{
"profilers": [
{
"name": "network",
"excludeHosts": ["telemetry.local", "monitoring.internal"]
}
]
}
```

#### Environment variable

```bash
export QASE_PROFILER_NETWORK_EXCLUDE_HOSTS="telemetry.local,monitoring.internal"
```

#### Combining both

The environment variable and config file values are independent — the environment variable sets the exclude list directly (it does not merge with the config file). The Qase API host (`testops.api.host`) is always excluded automatically regardless of this setting.

### Multiple Profilers

You can enable multiple profilers simultaneously, mixing string and object formats:

```json
{
"profilers": [
{
"name": "network",
"excludeHosts": ["telemetry.local"]
},
"db",
"sleep"
]
}
```

## Collected Data

For each HTTP request, the profiler collects the following information:

### Request Information

- **method** — HTTP method (GET, POST, PUT, DELETE, etc.)
- **url** — Full request URL
- **body** — Request body (when available)
- **headers** — Request headers (when available)

### Response Information

- **status_code** — HTTP response status code
- **response_body** — Response body (only for failed requests with status >= 400)
- **response_headers** — Response headers (only for failed requests with status >= 400)

## Data Format in Qase TestOps

When sent to Qase TestOps, HTTP requests appear as test steps with:

- **Step type**: `request`
- **Status**: `passed` for status < 400, `failed` for status >= 400

## Automatic Integration

The network profiler works automatically once enabled in the configuration. No additional code changes are required — it intercepts HTTP operations transparently using monkey patching.

## Notes

- The Qase API host is always excluded from tracking to avoid recursive logging
- The profiler only tracks operations that occur after it's enabled
- Failed responses (status >= 400) include response body and headers for debugging
- The profiler handles errors gracefully and won't break your HTTP operations
5 changes: 2 additions & 3 deletions qase-python-commons/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "qase-python-commons"
version = "5.0.2"
version = "5.0.3"
description = "A library for Qase TestOps and Qase Report"
readme = "README.md"
authors = [{name = "Qase Team", email = "support@qase.io"}]
Expand Down Expand Up @@ -60,8 +60,7 @@ passenv =
commands =
pytest --cov-config=pyproject.toml {posargs}
extras =
all
testing
testing
"""

[tool.setuptools.packages.find]
Expand Down
5 changes: 5 additions & 0 deletions qase-python-commons/src/qase/commons/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,11 @@ def __load_env_config(self):
if key == 'QASE_PROFILERS':
self.config.set_profilers(value.split(','))

if key == 'QASE_PROFILER_NETWORK_EXCLUDE_HOSTS':
self.config.network_profiler.set_exclude_hosts(
[h.strip() for h in value.split(',')]
)

if key == 'QASE_DEBUG':
self.config.set_debug(value)

Expand Down
23 changes: 22 additions & 1 deletion qase-python-commons/src/qase/commons/models/config/qaseconfig.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,16 @@ def set_path(self, path: str):
self.path = path


class NetworkProfilerConfig(BaseModel):
exclude_hosts: List[str] = None

def __init__(self):
self.exclude_hosts = []

def set_exclude_hosts(self, hosts: List[str]):
self.exclude_hosts = hosts


class QaseConfig(BaseModel):
mode: Mode = None
fallback: Mode = None
Expand All @@ -51,6 +61,7 @@ class QaseConfig(BaseModel):
testops_multi: TestopsMultiConfig = None
report: ReportConfig = None
profilers: list = None
network_profiler: NetworkProfilerConfig = None
framework: Framework = None
exclude_params: list = None
status_mapping: Dict[str, str] = None
Expand All @@ -66,6 +77,7 @@ def __init__(self):
self.execution_plan = ExecutionPlan()
self.framework = Framework()
self.profilers = []
self.network_profiler = NetworkProfilerConfig()
self.exclude_params = []
self.status_mapping = {}
self.logging = LoggingConfig()
Expand All @@ -82,7 +94,16 @@ def set_environment(self, environment: str):
self.environment = environment

def set_profilers(self, profilers: list):
self.profilers = profilers
self.profilers = []
for item in profilers:
if isinstance(item, str):
self.profilers.append(item)
elif isinstance(item, dict):
name = item.get("name")
if name:
self.profilers.append(name)
if name == "network" and "excludeHosts" in item:
self.network_profiler.set_exclude_hosts(item["excludeHosts"])

def set_root_suite(self, root_suite: str):
self.root_suite = root_suite
Expand Down
26 changes: 16 additions & 10 deletions qase-python-commons/src/qase/commons/profilers/network.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,24 @@
import threading
import uuid
from functools import wraps
from typing import List
from ..models.runtime import Runtime
from ..models.step import Step, StepRequestData, StepType


class NetworkProfiler:
_instance = None

def __init__(self, runtime: Runtime, skip_domain: str, track_on_fail: bool = True):
def __init__(self, runtime: Runtime, skip_domains: List[str] = None, track_on_fail: bool = True):
self._original_functions = {}
self.runtime = runtime
self.skip_domain = skip_domain
self.skip_domains = skip_domains or []
self.track_on_fail = track_on_fail
self.step = None

def _should_skip(self, url: str) -> bool:
return any(domain in url for domain in self.skip_domains)

def enable(self):
if 'requests' in sys.modules:
import requests
Expand All @@ -39,29 +43,31 @@ def disable(self):
def _requests_send_wrapper(self, func):
@wraps(func)
def wrapper(self, request, *args, **kwargs):
NetworkProfilerSingleton.get_instance()._log_request(request)
profiler = NetworkProfilerSingleton.get_instance()
if profiler._should_skip(request.url):
return func(self, request, *args, **kwargs)

profiler._log_request(request)
response = func(self, request, *args, **kwargs)
NetworkProfilerSingleton.get_instance()._log_response(response)
profiler._log_response(response)
return response

return wrapper

def _urllib3_request_wrapper(self, func):
skip_domain = self.skip_domain

@wraps(func)
def wrapper(self, method, url, *args, **kwargs):
if skip_domain in url:
profiler = NetworkProfilerSingleton.get_instance()
if profiler._should_skip(url):
return func(self, method, url, *args, **kwargs)

interceptor = NetworkProfilerSingleton.get_instance()
request = lambda: None
request.method = method
request.url = url

interceptor._log_request(request)
profiler._log_request(request)
response = func(self, method, url, *args, **kwargs)
interceptor._log_response(response, url=url)
profiler._log_response(response, url=url)
return response

return wrapper
Expand Down
4 changes: 3 additions & 1 deletion qase-python-commons/src/qase/commons/reporters/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -154,8 +154,10 @@ def setup_profilers(self, runtime: Runtime) -> None:
if profiler == "network":
# Lazy import
from ..profilers import NetworkProfilerSingleton
skip_domains = [self.config.testops.api.host]
skip_domains.extend(self.config.network_profiler.exclude_hosts)
NetworkProfilerSingleton.init(runtime=runtime,
skip_domain=self.config.testops.api.host)
skip_domains=skip_domains)
self.profilers.append(NetworkProfilerSingleton.get_instance())
if profiler == "sleep":
from ..profilers import SleepProfiler
Expand Down
Loading