Implement String Ranking Engine with Configurable Scoring System

[unclesp1d3r]

Summary

Create a flexible RankingEngine that assigns importance scores to extracted strings based on their semantic tags, source location, and section characteristics. This enables prioritization of potentially interesting strings in binary analysis.

Background

The string analyzer extracts and classifies strings from binaries with semantic tags (URLs, IPs, file paths, etc.), section types (code, data, resources), and source locations (imports, exports, section data). However, not all strings are equally interesting for analysis. A ranking system is needed to:

• Prioritize high-value strings (e.g., network indicators, file paths, registry keys)
• Deprioritize noise (e.g., common debug strings, version info)
• Weight by context (strings from executable sections vs. debug sections)
• Enable customizable scoring for different analysis scenarios (malware analysis, reverse engineering, compliance scanning)

Proposed Solution

Architecture

Create src/classification/ranking.rs with the following components:

1. RankingEngine struct: Main scoring engine with configurable weights
2. ScoreConfig struct: Configuration for tag weights, source weights, and section type multipliers
3. StringScore struct: Returned score with breakdown for transparency
4. Default scoring profiles: Presets for common use cases (malware analysis, general strings, etc.)

Scoring Algorithm

final_score = (tag_weight + source_weight) × section_type_multiplier

Tag Weights (base importance):

• High value (8-10): URLs, Domains, IPv4/IPv6, Email, Registry paths
• Medium value (5-7): File paths, GUIDs, Base64 (potential encoding)
• Lower value (2-4): Format strings, User agents
• Contextual (variable): Imports/Exports (depends on name), Version strings

Source Weights:

• ImportName/ExportName: +3 (API calls are interesting)
• SectionData: +2 (hardcoded strings)
• ResourceString: +1 (UI strings, less critical)
• DebugInfo: -2 (usually noise)

Section Type Multipliers:

• Code sections: ×1.5 (strings in executable code are unusual)
• StringData/ReadOnlyData: ×1.0 (expected location)
• WritableData: ×1.2 (potentially modified at runtime)
• Resources: ×0.8 (often benign UI strings)
• Debug: ×0.3 (low priority noise)

Implementation Details

pub struct RankingEngine {
    config: ScoreConfig,
}

pub struct ScoreConfig {
    tag_weights: HashMap<Tag, f32>,
    source_weights: HashMap<StringSource, f32>,
    section_multipliers: HashMap<SectionType, f32>,
}

pub struct StringScore {
    pub total: f32,
    pub tag_weight: f32,
    pub source_weight: f32,
    pub section_multiplier: f32,
}

impl RankingEngine {
    pub fn new(config: ScoreConfig) -> Self;
    pub fn with_defaults() -> Self;
    pub fn score(&self, tag: &Tag, source: StringSource, section: SectionType) -> StringScore;
}

Acceptance Criteria

• RankingEngine struct created with configurable scoring
• ScoreConfig supports custom weights for tags, sources, and sections
• Default scoring profile implemented with sensible weights
• score() method returns detailed StringScore with breakdown
• Unit tests for various scoring combinations
• Documentation with examples of customization
• Integration point in classification module (mod.rs)

Technical Notes

• Use f32 for scores to allow fractional weights
• Consider using builder pattern for ScoreConfig customization
• Scores should be normalized (0-100 range recommended)
• Future enhancement: Machine learning-based weight tuning

Dependencies

• Requires existing types from src/classification/mod.rs: Tag, StringSource, SectionType
• No external crate dependencies expected for MVP

References

• Requirements: 5.1
• Task-ID: stringy-analyzer/ranking-system-foundation

[unclesp1d3r]

@coderabbitai Enhance this issue body with additional context and a proposed solution. Improve the issue title and assign the proper labels and other metadata.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Read the current issue content
gh issue view 21 --json title,body,labels,assignees,milestone,state --repo EvilBit-Labs/StringyMcStringFace

Length of output: 1223

---

🏁 Script executed:

#!/bin/bash
# Get available labels in the repository to see what other labels we could use
gh label list --repo EvilBit-Labs/StringyMcStringFace --json name,description --limit 100

# Also check the codebase structure to provide better context
fd -t f -e rs . src/ | head -20

Length of output: 1098

---

🏁 Script executed:

#!/bin/bash
# Check the classification module to understand current structure
cat src/classification/mod.rs

# Check if there are any related types that might be relevant
cat src/types.rs | head -100

Length of output: 2488

---

🏁 Script executed:

#!/bin/bash
# Update the issue with enhanced content
gh issue edit 21 --repo EvilBit-Labs/StringyMcStringFace \
  --title "Implement String Ranking Engine with Configurable Scoring System" \
  --body "## Summary

Create a flexible RankingEngine that assigns importance scores to extracted strings based on their semantic tags, source location, and section characteristics. This enables prioritization of potentially interesting strings in binary analysis.

## Background

The string analyzer extracts and classifies strings from binaries with semantic tags (URLs, IPs, file paths, etc.), section types (code, data, resources), and source locations (imports, exports, section data). However, not all strings are equally interesting for analysis. A ranking system is needed to:

- **Prioritize high-value strings** (e.g., network indicators, file paths, registry keys)
- **Deprioritize noise** (e.g., common debug strings, version info)
- **Weight by context** (strings from executable sections vs. debug sections)
- **Enable customizable scoring** for different analysis scenarios (malware analysis, reverse engineering, compliance scanning)

## Proposed Solution

### Architecture

Create \`src/classification/ranking.rs\` with the following components:

1. **RankingEngine struct**: Main scoring engine with configurable weights
2. **ScoreConfig struct**: Configuration for tag weights, source weights, and section type multipliers
3. **StringScore struct**: Returned score with breakdown for transparency
4. **Default scoring profiles**: Presets for common use cases (malware analysis, general strings, etc.)

### Scoring Algorithm

\`\`\`
final_score = (tag_weight + source_weight) × section_type_multiplier
\`\`\`

**Tag Weights** (base importance):
- High value (8-10): URLs, Domains, IPv4/IPv6, Email, Registry paths
- Medium value (5-7): File paths, GUIDs, Base64 (potential encoding)
- Lower value (2-4): Format strings, User agents
- Contextual (variable): Imports/Exports (depends on name), Version strings

**Source Weights**:
- ImportName/ExportName: +3 (API calls are interesting)
- SectionData: +2 (hardcoded strings)
- ResourceString: +1 (UI strings, less critical)
- DebugInfo: -2 (usually noise)

**Section Type Multipliers**:
- Code sections: ×1.5 (strings in executable code are unusual)
- StringData/ReadOnlyData: ×1.0 (expected location)
- WritableData: ×1.2 (potentially modified at runtime)
- Resources: ×0.8 (often benign UI strings)
- Debug: ×0.3 (low priority noise)

### Implementation Details

\`\`\`rust
pub struct RankingEngine {
    config: ScoreConfig,
}

pub struct ScoreConfig {
    tag_weights: HashMap<Tag, f32>,
    source_weights: HashMap<StringSource, f32>,
    section_multipliers: HashMap<SectionType, f32>,
}

pub struct StringScore {
    pub total: f32,
    pub tag_weight: f32,
    pub source_weight: f32,
    pub section_multiplier: f32,
}

impl RankingEngine {
    pub fn new(config: ScoreConfig) -> Self;
    pub fn with_defaults() -> Self;
    pub fn score(&self, tag: &Tag, source: StringSource, section: SectionType) -> StringScore;
}
\`\`\`

## Acceptance Criteria

- [ ] \`RankingEngine\` struct created with configurable scoring
- [ ] \`ScoreConfig\` supports custom weights for tags, sources, and sections
- [ ] Default scoring profile implemented with sensible weights
- [ ] \`score()\` method returns detailed \`StringScore\` with breakdown
- [ ] Unit tests for various scoring combinations
- [ ] Documentation with examples of customization
- [ ] Integration point in classification module (\`mod.rs\`)

## Technical Notes

- Use \`f32\` for scores to allow fractional weights
- Consider using builder pattern for \`ScoreConfig\` customization
- Scores should be normalized (0-100 range recommended)
- Future enhancement: Machine learning-based weight tuning

## Dependencies

- Requires existing types from \`src/classification/mod.rs\`: \`Tag\`, \`StringSource\`, \`SectionType\`
- No external crate dependencies expected for MVP

## References

- Requirements: 5.1
- Task-ID: stringy-analyzer/ranking-system-foundation"

Length of output: 61

---

@unclesp1d3r I've successfully enhanced issue #21 with comprehensive details:

Changes Made:

Title Updated:

• From: "Ranking System Foundation"
• To: "Implement String Ranking Engine with Configurable Scoring System"

Enhanced Issue Body Includes:

1. Background Context - Explains why a ranking system is needed for prioritizing extracted strings in binary analysis

2. Proposed Solution with:

   • Complete architecture design (RankingEngine, ScoreConfig, StringScore structs)
   • Detailed scoring algorithm: final_score = (tag_weight + source_weight) × section_type_multiplier
   • Specific weight recommendations for different tag types, sources, and section types
   • Rust implementation outline with struct definitions

3. Acceptance Criteria - 7 checkboxes covering implementation, testing, and documentation requirements

4. Technical Notes - Implementation guidance (use f32, builder pattern, score normalization)

5. Dependencies - Clear references to existing types in the codebase

Labels & Metadata:
The existing labels are appropriate and have been retained:

• area:analyzer, type:enhancement, status:backlog, lang:rust, needs:tests
• Assigned to you
• Part of milestone v0.1

The issue is now ready for implementation with clear technical direction and acceptance criteria. You can view it at: #21