Core String Extraction Framework: StringExtractor Trait and Configuration

[unclesp1d3r]

Background

StringyMcStringFace aims to be a smarter alternative to the strings command by leveraging format-specific knowledge to extract and prioritize meaningful strings from binaries. The container parsing layer (ELF, PE, Mach-O) is already implemented and provides structured metadata via ContainerInfo. Now we need the core extraction framework that will scan binary data and produce FoundString instances.

Current State

• ✅ Container parsers implemented (src/container/)
• ✅ Type definitions complete (FoundString, Encoding, SectionType, etc.)
• ❌ String extraction logic missing (src/extraction/mod.rs is empty)
• ❌ No extraction configuration framework
• ❌ No trait abstraction for different extraction strategies

Objectives

Create a flexible, trait-based extraction framework in src/extraction/mod.rs that:

1. Defines the StringExtractor trait - An interface for different extraction strategies (basic ASCII, Unicode, context-aware, etc.)
2. Implements ExtractionConfig - Configurable parameters for extraction behavior
3. Provides a default extractor - A basic implementation to get started

Proposed Solution

1. StringExtractor Trait

/// Trait for implementing string extraction strategies
pub trait StringExtractor {
    /// Extract strings from binary data with metadata context
    fn extract(
        &self,
        data: &[u8],
        container_info: &ContainerInfo,
        config: &ExtractionConfig,
    ) -> Result<Vec<FoundString>>;
    
    /// Extract strings from a specific section
    fn extract_from_section(
        &self,
        data: &[u8],
        section: &SectionInfo,
        config: &ExtractionConfig,
    ) -> Result<Vec<FoundString>>;
}

2. ExtractionConfig Structure

/// Configuration parameters for string extraction
#[derive(Debug, Clone)]
pub struct ExtractionConfig {
    /// Minimum string length to consider
    pub min_length: usize,
    
    /// Maximum string length to extract
    pub max_length: usize,
    
    /// Encodings to search for
    pub encodings: Vec<Encoding>,
    
    /// Whether to extract from executable sections
    pub scan_code_sections: bool,
    
    /// Whether to include strings from debug sections
    pub include_debug: bool,
    
    /// Section types to prioritize
    pub section_priority: Vec<SectionType>,
    
    /// Whether to include import/export names
    pub include_symbols: bool,
}

impl Default for ExtractionConfig {
    fn default() -> Self {
        Self {
            min_length: 4,
            max_length: 4096,
            encodings: vec\![Encoding::Ascii, Encoding::Utf8],
            scan_code_sections: true,
            include_debug: false,
            section_priority: vec\![
                SectionType::StringData,
                SectionType::ReadOnlyData,
                SectionType::Resources,
            ],
            include_symbols: true,
        }
    }
}

3. Basic Implementation

Create a BasicExtractor that implements StringExtractor with straightforward sequential scanning for ASCII and UTF-8 strings.

Architecture Integration

The extraction framework fits into the pipeline as follows:

Binary Data → Container Parser → ContainerInfo
                                        ↓
                                  StringExtractor + ExtractionConfig
                                        ↓
                                  Vec<FoundString>
                                        ↓
                                  Classification (future)
                                        ↓
                                  Output Formatting

Implementation Details

File Structure

• Main trait and config: src/extraction/mod.rs
• Basic extractor: src/extraction/basic.rs (or inline in mod.rs initially)
• Tests: Unit tests in each module, integration tests in tests/

Key Considerations

• Use existing FoundString type (already defined in src/types.rs)
• Leverage ContainerInfo to make format-aware decisions
• Ensure zero-copy where possible (use slices and indices)
• Handle encoding detection robustly (invalid sequences should not panic)
• Set appropriate StringSource values based on where strings are found

Acceptance Criteria

• StringExtractor trait defined with documented methods
• ExtractionConfig struct with sensible defaults
• At least one working implementation (BasicExtractor)
• Unit tests covering:
  • ASCII string extraction
  • UTF-8 string extraction
  • Configuration validation
  • Edge cases (empty data, invalid encodings, boundary conditions)
• Documentation with usage examples
• Integration with existing ContainerInfo architecture

Related

• Requirement: 2.1 (String Extraction Framework)
• Task ID: stringy-analyzer/string-extraction-framework
• Blocks: Classification and scoring functionality (future issues)
• Repository: https://github.com/EvilBit-Labs/StringyMcStringFace

Notes

• The issue originally mentioned RawString struct, but FoundString already exists in src/types.rs and should be used instead
• Future extractors can implement format-specific logic (e.g., PE resource strings, Mach-O LC_ID_DYLIB)
• This is the foundation for the entire extraction pipeline - needs to be solid and extensible

[unclesp1d3r]

@coderabbitai Enhance this issue body with additional context and a proposed solution. Improve the issue title and assign the proper labels and other metadata.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Fetch issue #8 details
gh issue view 8 --json title,body,labels,assignees,state,url

Length of output: 1167

---

🏁 Script executed:

#!/bin/bash
# Explore the project structure to understand context
echo "=== Project Structure ==="
fd -t d -d 2

echo -e "\n=== Check if extraction module exists ==="
fd -e rs extraction

echo -e "\n=== Look for existing string extraction code ==="
rg -n "string.*extract" -i --type rust -C 2

echo -e "\n=== Check main crate structure ==="
cat Cargo.toml 2>/dev/null || echo "Cargo.toml not found"

echo -e "\n=== Check src directory structure ==="
fd -e rs . src 2>/dev/null | head -20

Length of output: 3214

---

🏁 Script executed:

#!/bin/bash
# Read key files to understand the architecture
echo "=== src/extraction/mod.rs ==="
cat src/extraction/mod.rs

echo -e "\n=== src/types.rs ==="
cat src/types.rs

echo -e "\n=== src/container/mod.rs ==="
cat src/container/mod.rs

echo -e "\n=== src/lib.rs (to understand module structure) ==="
cat src/lib.rs

Length of output: 7810

---

🏁 Script executed:

#!/bin/bash
# Update issue #8 with enhanced content
gh issue edit 8 \
  --title "Core String Extraction Framework: StringExtractor Trait and Configuration" \
  --body "## Background

StringyMcStringFace aims to be a smarter alternative to the \`strings\` command by leveraging format-specific knowledge to extract and prioritize meaningful strings from binaries. The container parsing layer (ELF, PE, Mach-O) is already implemented and provides structured metadata via \`ContainerInfo\`. Now we need the core extraction framework that will scan binary data and produce \`FoundString\` instances.

## Current State

- ✅ Container parsers implemented (\`src/container/\`)
- ✅ Type definitions complete (\`FoundString\`, \`Encoding\`, \`SectionType\`, etc.)
- ❌ String extraction logic missing (\`src/extraction/mod.rs\` is empty)
- ❌ No extraction configuration framework
- ❌ No trait abstraction for different extraction strategies

## Objectives

Create a flexible, trait-based extraction framework in \`src/extraction/mod.rs\` that:

1. **Defines the \`StringExtractor\` trait** - An interface for different extraction strategies (basic ASCII, Unicode, context-aware, etc.)
2. **Implements \`ExtractionConfig\`** - Configurable parameters for extraction behavior
3. **Provides a default extractor** - A basic implementation to get started

## Proposed Solution

### 1. StringExtractor Trait

\`\`\`rust
/// Trait for implementing string extraction strategies
pub trait StringExtractor {
    /// Extract strings from binary data with metadata context
    fn extract(
        &self,
        data: &[u8],
        container_info: &ContainerInfo,
        config: &ExtractionConfig,
    ) -> Result<Vec<FoundString>>;
    
    /// Extract strings from a specific section
    fn extract_from_section(
        &self,
        data: &[u8],
        section: &SectionInfo,
        config: &ExtractionConfig,
    ) -> Result<Vec<FoundString>>;
}
\`\`\`

### 2. ExtractionConfig Structure

\`\`\`rust
/// Configuration parameters for string extraction
#[derive(Debug, Clone)]
pub struct ExtractionConfig {
    /// Minimum string length to consider
    pub min_length: usize,
    
    /// Maximum string length to extract
    pub max_length: usize,
    
    /// Encodings to search for
    pub encodings: Vec<Encoding>,
    
    /// Whether to extract from executable sections
    pub scan_code_sections: bool,
    
    /// Whether to include strings from debug sections
    pub include_debug: bool,
    
    /// Section types to prioritize
    pub section_priority: Vec<SectionType>,
    
    /// Whether to include import/export names
    pub include_symbols: bool,
}

impl Default for ExtractionConfig {
    fn default() -> Self {
        Self {
            min_length: 4,
            max_length: 4096,
            encodings: vec![Encoding::Ascii, Encoding::Utf8],
            scan_code_sections: true,
            include_debug: false,
            section_priority: vec![
                SectionType::StringData,
                SectionType::ReadOnlyData,
                SectionType::Resources,
            ],
            include_symbols: true,
        }
    }
}
\`\`\`

### 3. Basic Implementation

Create a \`BasicExtractor\` that implements \`StringExtractor\` with straightforward sequential scanning for ASCII and UTF-8 strings.

## Architecture Integration

The extraction framework fits into the pipeline as follows:

\`\`\`
Binary Data → Container Parser → ContainerInfo
                                        ↓
                                  StringExtractor + ExtractionConfig
                                        ↓
                                  Vec<FoundString>
                                        ↓
                                  Classification (future)
                                        ↓
                                  Output Formatting
\`\`\`

## Implementation Details

### File Structure
- Main trait and config: \`src/extraction/mod.rs\`
- Basic extractor: \`src/extraction/basic.rs\` (or inline in mod.rs initially)
- Tests: Unit tests in each module, integration tests in \`tests/\`

### Key Considerations
- Use existing \`FoundString\` type (already defined in \`src/types.rs\`)
- Leverage \`ContainerInfo\` to make format-aware decisions
- Ensure zero-copy where possible (use slices and indices)
- Handle encoding detection robustly (invalid sequences should not panic)
- Set appropriate \`StringSource\` values based on where strings are found

## Acceptance Criteria

- [ ] \`StringExtractor\` trait defined with documented methods
- [ ] \`ExtractionConfig\` struct with sensible defaults
- [ ] At least one working implementation (\`BasicExtractor\`)
- [ ] Unit tests covering:
  - ASCII string extraction
  - UTF-8 string extraction  
  - Configuration validation
  - Edge cases (empty data, invalid encodings, boundary conditions)
- [ ] Documentation with usage examples
- [ ] Integration with existing \`ContainerInfo\` architecture

## Related

- **Requirement**: 2.1 (String Extraction Framework)
- **Task ID**: stringy-analyzer/string-extraction-framework
- **Blocks**: Classification and scoring functionality (future issues)
- **Repository**: https://github.com/EvilBit-Labs/StringyMcStringFace

## Notes

- The issue originally mentioned \`RawString\` struct, but \`FoundString\` already exists in \`src/types.rs\` and should be used instead
- Future extractors can implement format-specific logic (e.g., PE resource strings, Mach-O LC_ID_DYLIB)
- This is the foundation for the entire extraction pipeline - needs to be solid and extensible"

echo "Issue updated successfully"

Length of output: 87

---

🏁 Script executed:

#!/bin/bash
# Check available labels in the repository
gh label list --limit 100

echo -e "\n=== Current issue labels ==="
gh issue view 8 --json labels --jq '.labels[].name'

Length of output: 741

---

@unclesp1d3r I've successfully enhanced issue #8 with comprehensive details:

Changes Made

✅ Title Updated

Changed from "String Extraction Framework" to "Core String Extraction Framework: StringExtractor Trait and Configuration" for better clarity and specificity.

✅ Body Enhanced With

1. Background & Context - Explains how this fits into StringyMcStringFace's architecture and relates to existing container parsers
2. Current State - Clear checklist showing what's done (parsers, types) and what's missing (extraction logic)
3. Detailed Objectives - Three clear goals for the trait, config, and default implementation
4. Proposed Solution - Complete API designs with Rust code examples for:
   • StringExtractor trait with two methods
   • ExtractionConfig struct with sensible defaults (min_length: 4, encodings, section priorities, etc.)
   • Suggestion for a BasicExtractor implementation
5. Architecture Diagram - Visual pipeline showing data flow
6. Implementation Details - File structure, key considerations (zero-copy, encoding handling, etc.)
7. Acceptance Criteria - Testable checklist including unit tests for ASCII, UTF-8, edge cases
8. Notes - Clarified that FoundString (not RawString) should be used, and mentioned extensibility for future format-specific extractors

✅ Labels & Metadata

Existing labels are perfect and remain unchanged:

• area:analyzer - Correct functional area
• type:enhancement - Appropriate type
• status:backlog - Current status
• lang:rust - Implementation language
• needs:tests - Correctly reflects acceptance criteria

The issue is now ready for implementation with clear technical specifications and acceptance criteria!