Skip to content

feat: add support for configurable acceptable thresholds in octocov output #29

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
k1LoW merged 1 commit into from
Sep 17, 2025
Merged

feat: add support for configurable acceptable thresholds in octocov output #29

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 55 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ export TAILOR_TOKEN=your_access_token
patterner metrics
```

The metrics command displays metrics in a table format. Use the `--out-octocov-path` option to output metrics in octocov custom metrics format.
The metrics command displays metrics in a table format. Use the `--out-octocov-path` option to output metrics in octocov custom metrics format with configurable acceptable thresholds.

#### Metrics Options

Expand Down Expand Up @@ -124,6 +124,43 @@ patterner metrics --out-octocov-path metrics.json
patterner metrics --since 24hours --out-octocov-path metrics.json --with-coverage-full-report --with-lint-warnings
```

#### Octocov Custom Metrics Format

When using the `--out-octocov-path` option, patterner outputs metrics in octocov custom metrics format. This format includes:

- **Metrics data** - All collected metrics with their values and units
- **Acceptable thresholds** - Configurable conditions that define acceptable metric values
- Configured via `metrics.octocov.acceptables` in `.patterner.yml`
- Used by [octocov](https://github.com/k1LoW/octocov) to evaluate whether metrics meet quality standards

**Example octocov output:**
```json
[
{
"key": "patterner-metrics",
"name": "Patterner Metrics",
"metrics": [
{
"key": "pipeline_resolver_step_coverage_percentage",
"name": "Pipeline Resolver Step Coverage",
"value": 75.0,
"unit": "%"
},
{
"key": "lint_warnings_total",
"name": "Lint Warnings Total",
"value": 3,
"unit": "count"
}
],
"acceptables": [
"pipeline_resolver_step_coverage_percentage >= 80",
"lint_warnings_total <= 5"
]
}
]
```

**Implementation Notes:**
```
<!-- Note: Current implementation has a typo in variable name 'lint_warnigns_total'
Expand Down Expand Up @@ -252,8 +289,25 @@ lint:
stateflow:
deprecatedFeature:
enabled: true
metrics:
octocov:
acceptables:
- "current.pipeline_resolver_step_coverage_percentage >= 80"
- "diff.lint_warnings_total <= 0"
```

### Metrics Configuration

#### Octocov Integration

- **octocov** - Configuration for octocov custom metrics format output
- `acceptables` - List of acceptable threshold conditions for metrics
- Define minimum acceptable values for metrics in octocov format
- Used when outputting metrics with `--out-octocov-path` option
- Format: `"metric_name operator value"` (e.g., `"coverage_percentage >= 80"`)
- Supports various metrics including coverage percentages and lint warning counts
- Example: `["pipeline_resolver_step_coverage_percentage >= 80", "lint_warnings_total <= 5"]`

### Lint Configuration

#### Acceptable Warnings
Expand Down
11 changes: 7 additions & 4 deletions cmd/metrics.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,8 @@ var metricsCmd = &cobra.Command{
Unit: m.Unit,
})
}
metricSet.Acceptables = append(metricSet.Acceptables, cfg.Metrics.Octocov.Acceptables...)

csets := []*CustomMetricSet{metricSet}
b, err := json.MarshalIndent(csets, "", " ")
if err != nil {
Expand Down Expand Up @@ -219,10 +221,11 @@ type MetadataKV struct {
}

type CustomMetricSet struct {
Key string `json:"key"`
Name string `json:"name,omitempty"`
Metadata []*MetadataKV `json:"metadata,omitempty"`
Metrics []*CustomMetric `json:"metrics"`
Key string `json:"key"`
Name string `json:"name,omitempty"`
Metadata []*MetadataKV `json:"metadata,omitempty"`
Metrics []*CustomMetric `json:"metrics"`
Acceptables []string `json:"acceptables,omitempty"`
}

type CustomMetric struct {
Expand Down
15 changes: 12 additions & 3 deletions config/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@ import (
)

type Config struct {
WorkspaceID string `default:"" yaml:"workspaceID,omitempty"`
Lint Lint `yaml:"lint,omitempty"`
WorkspaceID string `default:"" yaml:"workspaceID,omitempty"`
Lint Lint `yaml:"lint,omitempty"`
Metrics Metrics `yaml:"metrics,omitempty"`
}

type Lint struct {
Expand All @@ -27,7 +28,7 @@ type Rules struct {
type Pipeline struct {
DeprecatedFeature PipelineDeprecatedFeature `yaml:"deprecatedFeature,omitempty,omitzero"`
InsecureAuthorization InsecureAuthorization `yaml:"insecureAuthorization,omitempty,omitzero"`
StepCount StepCount `yaml:"stepCount,omitempty,omitzero"`
StepCount StepCount `yaml:"stepCount,omitempty,omitzero"`
MultipleMutations MultipleMutations `yaml:"multipleMutations,omitempty,omitzero"`
QueryBeforeMutation QueryBeforeMutation `yaml:"queryBeforeMutation,omitempty,omitzero"`
}
Expand Down Expand Up @@ -76,6 +77,14 @@ type StateFlowDeprecatedFeature struct {
Enabled bool `default:"true" yaml:"enabled,omitempty"`
}

type Metrics struct {
Octocov Octocov `yaml:"octocov,omitempty,omitzero"`
Comment thread
k1LoW marked this conversation as resolved.
}

type Octocov struct {
Acceptables []string `yaml:"acceptables,omitempty"`
}

const Filename = ".patterner.yml"

func New() (*Config, error) {
Expand Down