Integrate Import/Export Symbol Extraction into String Analysis Pipeline

[unclesp1d3r]

Overview

Complete the integration of import/export symbol extraction into the main string extraction and ranking pipeline. The foundation already exists in the container parsers (ELF, PE, Mach-O), but these symbols need to be:

• Surfaced as searchable/filterable strings
• Classified with semantic tags for API categories
• Integrated into the scoring algorithm
• Exposed via CLI filtering options

Current Implementation Status

✅ Already Implemented

• Parser Infrastructure (src/container/{elf,pe,macho}.rs):
  • extract_imports() and extract_exports() methods exist
  • ELF: Reads .dynsym and .symtab sections
  • PE: Parses Import Directory Table and Export Address Table
  • Mach-O: Extracts from symbol table (undefined = imports, defined = exports)
• Data Structures (src/types.rs):
  • ImportInfo, ExportInfo, ContainerInfo
  • Tag::Import, Tag::Export enum variants
  • StringSource::ImportName, StringSource::ExportName

❌ Not Yet Implemented

• Main extraction pipeline to surface imports/exports as FoundString objects
• Semantic classification of API categories (crypto, network, file I/O, etc.)
• Integration into scoring/ranking algorithm with adjustable weights
• CLI flags: --imports, --exports, --symbols
• Test coverage for import/export extraction

Proposed Solution

1. Pipeline Integration (src/extraction/mod.rs)

Convert ImportInfo/ExportInfo from container parsers into FoundString objects:

• Set source field to StringSource::ImportName or ExportName
• Apply base tags: Tag::Import or Tag::Export
• Preserve library/ordinal metadata as context
• Calculate RVA/offset from address field

2. Semantic Classification (src/classification/mod.rs)

Implement API categorization based on symbol name patterns:

• Crypto APIs: AES*, RSA*, SHA*, CryptEncrypt, EVP_*, CCCrypt*
• Network APIs: socket, connect, recv, send, WSA*, getaddrinfo
• File I/O: fopen, CreateFile, ReadFile, open, read, write
• Process APIs: CreateProcess, execve, fork, WinExec
• Registry: RegOpenKey, RegSetValue, etc. (Windows-specific)

Add corresponding tag variants to Tag enum (e.g., Tag::CryptoApi, Tag::NetworkApi).

3. Ranking Integration

Add configurable weights to scoring algorithm:

• Base score for all imports/exports (e.g., +5)
• Bonus for suspicious/high-value APIs (e.g., +15 for crypto, +10 for network)
• Consider frequency (rare imports score higher)
• Configurable via scoring parameters

4. CLI Enhancement (src/main.rs)

Add filtering flags to clap parser:

#[arg(long, help = "Show only imported symbols")]
imports: bool,

#[arg(long, help = "Show only exported symbols")]
exports: bool,

#[arg(long, help = "Show all symbols (imports + exports)")]
symbols: bool,

Acceptance Criteria

• Imports from all three formats (ELF, PE, Mach-O) appear in extraction output
• Exports from all three formats appear in extraction output
• Import/export strings are tagged with Tag::Import or Tag::Export
• API category classification applied (crypto, network, file I/O, process, registry)
• Scoring algorithm incorporates import/export weights
• CLI flags --imports, --exports, --symbols filter output correctly
• Integration tests with real-world binaries:
  • ELF: Linux executable with glibc imports
  • PE: Windows DLL with kernel32/advapi32 imports
  • Mach-O: macOS binary with libSystem exports
• Unit tests for API classification logic
• Documentation updated with examples of import/export filtering

Implementation Notes

• Reuse existing parser methods—no changes needed to src/container/* files
• Focus on src/extraction/mod.rs for pipeline integration
• Consider adding a classify_api_category() helper in src/classification/mod.rs
• Scoring weights should be configurable (future-proof for config file support)
• Output format should clearly distinguish imports from exports (e.g., library name for imports)

Testing Strategy

1. Unit Tests: API pattern matching for crypto/network/file APIs
2. Integration Tests:
   • tests/integration_elf.rs: Verify libc.so.6 imports (e.g., malloc, fopen)
   • tests/integration_pe.rs: Verify kernel32.dll imports (e.g., CreateFileW)
   • tests/integration_macho.rs: Verify libSystem.B.dylib symbols
3. Snapshot Tests: Capture expected output with --imports and --exports flags

References

• Parent Epic: #40 - Epic: v0.2 - PE Resources, Symbol Demangling & Import/Export Enhancement
• Related: Depends on main extraction pipeline implementation
• Goblin Documentation: https://docs.rs/goblin/latest/goblin/

[unclesp1d3r]

@coderabbitai Enhance this issue body with additional context and a proposed solution. Improve the issue title and assign the proper labels and other metadata.

[traycerai]

Warning

Rate limit exceeded. Please try after 225 seconds.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash

# View the current issue #59
echo "=== Current Issue #59 ==="
gh issue view 59 --json title,body,labels,assignees,milestone,state | jq .

echo -e "\n=== Available Labels in Repository ==="
gh label list --limit 100

echo -e "\n=== Repository Structure ==="
fd -t d -d 2 | head -20

echo -e "\n=== Looking for parser-related files ==="
fd -e rs -e go -e py parser | head -20

echo -e "\n=== Looking for container/binary format parsers ==="
rg -l "elf|mach-o|macho|pe32" --type rust | head -10

Length of output: 3863

---

🏁 Script executed:

#!/bin/bash

echo "=== ELF Parser - Import/Export handling ==="
rg -n "import|export|symbol|dynsym|symtab" src/container/elf.rs -A 2 -B 2 | head -60

echo -e "\n=== PE Parser - Import/Export handling ==="
rg -n "import|export|Import.*Table|Export.*Table" src/container/pe.rs -A 2 -B 2 | head -60

echo -e "\n=== Mach-O Parser - Import/Export handling ==="
rg -n "import|export|symbol|LC_DYLD" src/container/macho.rs -A 2 -B 2 | head -60

echo -e "\n=== Types and Classification ==="
rg -n "enum.*Tag|struct.*Tag|Classification" src/types.rs -A 3 | head -40

echo -e "\n=== Extraction Module ==="
cat src/extraction/mod.rs | head -50

echo -e "\n=== CLI arguments ==="
rg -n "clap|Arg|import|export" src/main.rs -A 2 -B 2 | head -60

Length of output: 6842

---

🏁 Script executed:

#!/bin/bash

echo "=== Full Types Definition ==="
cat src/types.rs

echo -e "\n=== Main CLI Structure ==="
cat src/main.rs

echo -e "\n=== Classification Module ==="
cat src/classification/mod.rs 2>/dev/null || echo "Classification module not found"

echo -e "\n=== Check if imports/exports are used in extraction ==="
rg -n "ImportInfo|ExportInfo|imports|exports" src/extraction/ -C 2

Length of output: 5921

---

🏁 Script executed:

#!/bin/bash

# Update issue #59 with enhanced content
gh issue edit 59 \
  --title "Integrate Import/Export Symbol Extraction into String Analysis Pipeline" \
  --body "## Overview

Complete the integration of import/export symbol extraction into the main string extraction and ranking pipeline. The foundation already exists in the container parsers (ELF, PE, Mach-O), but these symbols need to be:
- Surfaced as searchable/filterable strings
- Classified with semantic tags for API categories
- Integrated into the scoring algorithm
- Exposed via CLI filtering options

## Current Implementation Status

### ✅ Already Implemented
- **Parser Infrastructure** (\`src/container/{elf,pe,macho}.rs\`):
  - \`extract_imports()\` and \`extract_exports()\` methods exist
  - ELF: Reads \`.dynsym\` and \`.symtab\` sections
  - PE: Parses Import Directory Table and Export Address Table
  - Mach-O: Extracts from symbol table (undefined = imports, defined = exports)
- **Data Structures** (\`src/types.rs\`):
  - \`ImportInfo\`, \`ExportInfo\`, \`ContainerInfo\`
  - \`Tag::Import\`, \`Tag::Export\` enum variants
  - \`StringSource::ImportName\`, \`StringSource::ExportName\`

### ❌ Not Yet Implemented
- Main extraction pipeline to surface imports/exports as \`FoundString\` objects
- Semantic classification of API categories (crypto, network, file I/O, etc.)
- Integration into scoring/ranking algorithm with adjustable weights
- CLI flags: \`--imports\`, \`--exports\`, \`--symbols\`
- Test coverage for import/export extraction

## Proposed Solution

### 1. Pipeline Integration (\`src/extraction/mod.rs\`)
Convert \`ImportInfo\`/\`ExportInfo\` from container parsers into \`FoundString\` objects:
- Set \`source\` field to \`StringSource::ImportName\` or \`ExportName\`
- Apply base tags: \`Tag::Import\` or \`Tag::Export\`
- Preserve library/ordinal metadata as context
- Calculate RVA/offset from address field

### 2. Semantic Classification (\`src/classification/mod.rs\`)
Implement API categorization based on symbol name patterns:
- **Crypto APIs**: \`AES*\`, \`RSA*\`, \`SHA*\`, \`CryptEncrypt\`, \`EVP_*\`, \`CCCrypt*\`
- **Network APIs**: \`socket\`, \`connect\`, \`recv\`, \`send\`, \`WSA*\`, \`getaddrinfo\`
- **File I/O**: \`fopen\`, \`CreateFile\`, \`ReadFile\`, \`open\`, \`read\`, \`write\`
- **Process APIs**: \`CreateProcess\`, \`execve\`, \`fork\`, \`WinExec\`
- **Registry**: \`RegOpenKey\`, \`RegSetValue\`, etc. (Windows-specific)

Add corresponding tag variants to \`Tag\` enum (e.g., \`Tag::CryptoApi\`, \`Tag::NetworkApi\`).

### 3. Ranking Integration
Add configurable weights to scoring algorithm:
- Base score for all imports/exports (e.g., +5)
- Bonus for suspicious/high-value APIs (e.g., +15 for crypto, +10 for network)
- Consider frequency (rare imports score higher)
- Configurable via scoring parameters

### 4. CLI Enhancement (\`src/main.rs\`)
Add filtering flags to \`clap\` parser:
\`\`\`rust
#[arg(long, help = \"Show only imported symbols\")]
imports: bool,

#[arg(long, help = \"Show only exported symbols\")]
exports: bool,

#[arg(long, help = \"Show all symbols (imports + exports)\")]
symbols: bool,
\`\`\`

## Acceptance Criteria

- [ ] Imports from all three formats (ELF, PE, Mach-O) appear in extraction output
- [ ] Exports from all three formats appear in extraction output
- [ ] Import/export strings are tagged with \`Tag::Import\` or \`Tag::Export\`
- [ ] API category classification applied (crypto, network, file I/O, process, registry)
- [ ] Scoring algorithm incorporates import/export weights
- [ ] CLI flags \`--imports\`, \`--exports\`, \`--symbols\` filter output correctly
- [ ] Integration tests with real-world binaries:
  - ELF: Linux executable with glibc imports
  - PE: Windows DLL with kernel32/advapi32 imports
  - Mach-O: macOS binary with libSystem exports
- [ ] Unit tests for API classification logic
- [ ] Documentation updated with examples of import/export filtering

## Implementation Notes

- Reuse existing parser methods—no changes needed to \`src/container/*\` files
- Focus on \`src/extraction/mod.rs\` for pipeline integration
- Consider adding a \`classify_api_category()\` helper in \`src/classification/mod.rs\`
- Scoring weights should be configurable (future-proof for config file support)
- Output format should clearly distinguish imports from exports (e.g., library name for imports)

## Testing Strategy

1. **Unit Tests**: API pattern matching for crypto/network/file APIs
2. **Integration Tests**: 
   - \`tests/integration_elf.rs\`: Verify \`libc.so.6\` imports (e.g., \`malloc\`, \`fopen\`)
   - \`tests/integration_pe.rs\`: Verify \`kernel32.dll\` imports (e.g., \`CreateFileW\`)
   - \`tests/integration_macho.rs\`: Verify \`libSystem.B.dylib\` symbols
3. **Snapshot Tests**: Capture expected output with \`--imports\` and \`--exports\` flags

## References

- Parent Epic: [#40 - Epic: v0.2 - PE Resources, Symbol Demangling & Import/Export Enhancement](https://github.com/EvilBit-Labs/StringyMcStringFace/issues/40)
- Related: Depends on main extraction pipeline implementation
- Goblin Documentation: https://docs.rs/goblin/latest/goblin/"

# Add labels
gh issue edit 59 --add-label "area:analyzer,lang:rust,needs:tests,priority:medium"

echo "✅ Issue #59 updated successfully"

Length of output: 156

---

@unclesp1d3r I've successfully enhanced issue #59! Here's what was updated:

📝 Title Enhancement

Before: "Import/Export Names Enhancement Integration"
After: "Integrate Import/Export Symbol Extraction into String Analysis Pipeline"

The new title is more specific and action-oriented, making it clearer what needs to be implemented.

🔍 Body Enhancements

Added comprehensive technical context including:

1. Current Implementation Status - Documented what's already working:

   • Parser infrastructure in src/container/{elf,pe,macho}.rs with extract_imports() and extract_exports() methods
   • Data structures (ImportInfo, ExportInfo, tags, string sources)
   • Clearly separated ✅ implemented vs. ❌ not implemented features

2. Detailed Proposed Solution with 4 key components:

   • Pipeline integration approach for converting imports/exports to FoundString objects
   • Semantic classification for API categories (crypto, network, file I/O, process, registry)
   • Ranking integration with configurable scoring weights
   • CLI enhancement with --imports, --exports, --symbols flags (including code examples)

3. Enhanced Acceptance Criteria - Expanded with specific test scenarios for each binary format

4. Implementation Notes - Guidance on reusing existing parsers and focusing on pipeline integration

5. Testing Strategy - Detailed unit, integration, and snapshot test approach with specific file references

🏷️ Labels Added

• area:analyzer - Binary analyzer functionality
• lang:rust - Rust implementation
• needs:tests - Requires test coverage
• priority:medium - Medium priority task

📊 Preserved Metadata

• ✅ Milestone: v0.2
• ✅ Assignee: unclesp1d3r
• ✅ Existing enhancement label

The issue is now ready for implementation with clear technical guidance and acceptance criteria! You can view the updated issue here: #59