Enhance ELF Dynamic Symbol Extraction with Library Mapping and Comprehensive Symbol Classification

[unclesp1d3r]

Summary

Enhance the existing ELF import/export extraction to comprehensively parse the dynamic section, extract library dependencies (DT_NEEDED entries), map symbols to their originating libraries, and improve symbol classification beyond just functions.

Context

The current ELF parser in src/container/elf.rs provides basic import/export extraction by analyzing the dynamic symbol table (.dynsym). However, it has several limitations:

1. No library mapping: Imports don't identify which shared library they come from (DT_NEEDED entries are not parsed)
2. Limited symbol types: Only extracts function symbols (STT_FUNC), missing data objects, TLS variables, etc.
3. Incomplete dynamic section parsing: The dynamic section contains additional metadata (version requirements, symbol versioning, etc.) that isn't currently extracted
4. Missing symbol visibility: Doesn't account for symbol visibility (STV_DEFAULT, STV_HIDDEN, etc.)

This enhancement extends the existing functionality to provide more complete import/export analysis, matching the comprehensiveness of the PE parser implementation.

Technical Background

ELF Dynamic Section Structure:

• Contains DT_NEEDED entries that specify required shared libraries
• Includes DT_SYMTAB, DT_STRTAB, and DT_HASH/DT_GNU_HASH for symbol resolution
• May contain version information via DT_VERNEED, DT_VERDEF, and DT_VERSYM

Symbol Classification:

• Imports: Undefined symbols (SHN_UNDEF) that need to be resolved at link/load time
• Exports: Defined symbols with global/weak binding available for other modules
• Symbol types: Functions (STT_FUNC), objects (STT_OBJECT), TLS (STT_TLS), IFuncs (STT_GNU_IFUNC)

Proposed Solution

Implementation Steps

1. Parse DT_NEEDED entries from dynamic section

   • Extract all required library names
   • Build a mapping of libraries for symbol attribution

2. Enhance import extraction (extract_imports)

   • Keep current undefined symbol detection
   • Extend to handle all symbol types (not just STT_FUNC):
     • STT_OBJECT (data objects)
     • STT_TLS (thread-local storage)
     • STT_GNU_IFUNC (indirect functions)
   • Attempt to map symbols to libraries using version information when available
   • Preserve symbol visibility and binding information

3. Enhance export extraction (extract_exports)

   • Include all globally visible defined symbols
   • Add support for weak symbols (STB_WEAK)
   • Include symbol type information in ExportInfo
   • Filter out hidden symbols (STV_HIDDEN, STV_INTERNAL)

4. Extend data structures if needed

   • May need to extend ImportInfo to include symbol type and version
   • May need to extend ExportInfo to include symbol type and binding

5. Add comprehensive unit tests

   • Test DT_NEEDED extraction with mock ELF data
   • Test symbol classification for various types
   • Test edge cases (weak symbols, versioned symbols, hidden symbols)
   • Use insta snapshots for symbol extraction results

6. Add integration tests with real binaries

   • Test with sample ELF binaries (e.g., /bin/ls equivalent)
   • Verify correct library mapping
   • Validate symbol counts and types

Code Structure

// New helper method
fn extract_needed_libraries(&self, elf: &Elf) -> Vec<String> {
    // Parse DT_NEEDED entries from elf.dynamic
}

// Enhanced import extraction
fn extract_imports(&self, elf: &Elf, libraries: &[String]) -> Vec<ImportInfo> {
    // Current logic + handle all symbol types + library mapping
}

// Enhanced export extraction  
fn extract_exports(&self, elf: &Elf) -> Vec<ExportInfo> {
    // Current logic + handle weak symbols + filter hidden
}

Requirements

4.2, 4.3

Acceptance Criteria

• ✅ Parse DT_NEEDED entries and extract library dependencies
• ✅ Map imported symbols to their originating libraries where possible
• ✅ Extract all relevant symbol types (functions, objects, TLS, IFuncs)
• ✅ Properly classify symbols as imports vs exports based on definition and binding
• ✅ Handle weak symbols and symbol visibility correctly
• ✅ Add comprehensive unit tests with cargo test passing
• ✅ Add integration tests with real ELF binaries
• ✅ Use insta for snapshot testing of symbol extraction
• ✅ Ensure cargo clippy -- -D warnings passes
• ✅ Update documentation in docs/ explaining the enhanced extraction
• ✅ Add benchmarks with criterion if performance-sensitive paths are added
• ✅ Update justfile recipes if new test targets are added
• ✅ Ensure CI passes

Dependencies

• Blocked by: Implement Intelligent ELF Section Classification for Targeted String Extraction #1 ELF Section Classification (appears to be already implemented based on codebase)

Task-ID

stringy-analyzer/elf-import-export-extraction

[unclesp1d3r]

@coderabbitai Enhance this issue body with additional context and a proposed solution. Improve the issue title and assign the proper labels and other metadata.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Fetch the details of issue #2
gh issue view 2 --json title,body,labels,assignees,state,url

# Also get repository information to understand the project context
gh repo view --json name,description,languages

# Get a sense of the codebase structure
fd -t f -e rb -e rake -e gemspec | head -20

Length of output: 2108

---

🏁 Script executed:

#!/bin/bash
# Look for existing ELF-related code to understand the current implementation
rg -n "ELF|elf" --type rust -C 2 | head -50

# Find any existing symbol or dynamic section handling
ast-grep --pattern 'struct $_ { $$$ }'  | rg -i "symbol|dynamic|elf" | head -20

# Check for any existing tests related to ELF
fd test -e rs --exec rg -l "elf|ELF" {} | head -10

# Look at the project structure
ls -la
find . -name "*.rs" -type f | head -20

Length of output: 3652

---

🏁 Script executed:

#!/bin/bash
# Examine the ELF parser implementation
cat src/container/elf.rs

# Look at the types to understand ImportInfo and ExportInfo structures
cat src/types.rs | grep -A 20 "struct ImportInfo\|struct ExportInfo\|struct SectionInfo"

# Check the container mod to understand the structure
cat src/container/mod.rs

Length of output: 10703

---

🏁 Script executed:

#!/bin/bash
# Update the issue with enhanced content
gh issue edit 2 \
  --title "Enhance ELF Dynamic Symbol Extraction with Library Mapping and Comprehensive Symbol Classification" \
  --body "## Summary

Enhance the existing ELF import/export extraction to comprehensively parse the dynamic section, extract library dependencies (DT_NEEDED entries), map symbols to their originating libraries, and improve symbol classification beyond just functions.

## Context

The current ELF parser in \`src/container/elf.rs\` provides basic import/export extraction by analyzing the dynamic symbol table (\`.dynsym\`). However, it has several limitations:

1. **No library mapping**: Imports don't identify which shared library they come from (DT_NEEDED entries are not parsed)
2. **Limited symbol types**: Only extracts function symbols (STT_FUNC), missing data objects, TLS variables, etc.
3. **Incomplete dynamic section parsing**: The dynamic section contains additional metadata (version requirements, symbol versioning, etc.) that isn't currently extracted
4. **Missing symbol visibility**: Doesn't account for symbol visibility (STV_DEFAULT, STV_HIDDEN, etc.)

This enhancement extends the existing functionality to provide more complete import/export analysis, matching the comprehensiveness of the PE parser implementation.

## Technical Background

**ELF Dynamic Section Structure:**
- Contains \`DT_NEEDED\` entries that specify required shared libraries
- Includes \`DT_SYMTAB\`, \`DT_STRTAB\`, and \`DT_HASH\`/\`DT_GNU_HASH\` for symbol resolution
- May contain version information via \`DT_VERNEED\`, \`DT_VERDEF\`, and \`DT_VERSYM\`

**Symbol Classification:**
- **Imports**: Undefined symbols (SHN_UNDEF) that need to be resolved at link/load time
- **Exports**: Defined symbols with global/weak binding available for other modules
- **Symbol types**: Functions (STT_FUNC), objects (STT_OBJECT), TLS (STT_TLS), IFuncs (STT_GNU_IFUNC)

## Proposed Solution

### Implementation Steps

1. **Parse DT_NEEDED entries from dynamic section**
   - Extract all required library names
   - Build a mapping of libraries for symbol attribution

2. **Enhance import extraction (\`extract_imports\`)**
   - Keep current undefined symbol detection
   - Extend to handle all symbol types (not just STT_FUNC):
     - STT_OBJECT (data objects)
     - STT_TLS (thread-local storage)
     - STT_GNU_IFUNC (indirect functions)
   - Attempt to map symbols to libraries using version information when available
   - Preserve symbol visibility and binding information

3. **Enhance export extraction (\`extract_exports\`)**
   - Include all globally visible defined symbols
   - Add support for weak symbols (STB_WEAK)
   - Include symbol type information in ExportInfo
   - Filter out hidden symbols (STV_HIDDEN, STV_INTERNAL)

4. **Extend data structures if needed**
   - May need to extend \`ImportInfo\` to include symbol type and version
   - May need to extend \`ExportInfo\` to include symbol type and binding

5. **Add comprehensive unit tests**
   - Test DT_NEEDED extraction with mock ELF data
   - Test symbol classification for various types
   - Test edge cases (weak symbols, versioned symbols, hidden symbols)
   - Use insta snapshots for symbol extraction results

6. **Add integration tests with real binaries**
   - Test with sample ELF binaries (e.g., \`/bin/ls\` equivalent)
   - Verify correct library mapping
   - Validate symbol counts and types

### Code Structure

\`\`\`rust
// New helper method
fn extract_needed_libraries(&self, elf: &Elf) -> Vec<String> {
    // Parse DT_NEEDED entries from elf.dynamic
}

// Enhanced import extraction
fn extract_imports(&self, elf: &Elf, libraries: &[String]) -> Vec<ImportInfo> {
    // Current logic + handle all symbol types + library mapping
}

// Enhanced export extraction  
fn extract_exports(&self, elf: &Elf) -> Vec<ExportInfo> {
    // Current logic + handle weak symbols + filter hidden
}
\`\`\`

## Requirements

4.2, 4.3

## Acceptance Criteria

- ✅ Parse DT_NEEDED entries and extract library dependencies
- ✅ Map imported symbols to their originating libraries where possible
- ✅ Extract all relevant symbol types (functions, objects, TLS, IFuncs)
- ✅ Properly classify symbols as imports vs exports based on definition and binding
- ✅ Handle weak symbols and symbol visibility correctly
- ✅ Add comprehensive unit tests with \`cargo test\` passing
- ✅ Add integration tests with real ELF binaries
- ✅ Use insta for snapshot testing of symbol extraction
- ✅ Ensure \`cargo clippy -- -D warnings\` passes
- ✅ Update documentation in \`docs/\` explaining the enhanced extraction
- ✅ Add benchmarks with criterion if performance-sensitive paths are added
- ✅ Update justfile recipes if new test targets are added
- ✅ Ensure CI passes

## Dependencies

- **Blocked by**: #1 ELF Section Classification (appears to be already implemented based on codebase)

## Task-ID

stringy-analyzer/elf-import-export-extraction"

# Add priority:medium label
gh issue edit 2 --add-label "priority:medium"

echo "Issue updated successfully!"

Length of output: 221

---

🏁 Script executed:

#!/bin/bash
# List all available labels in the repository
gh label list

# Check if there are any priority-related labels
gh label list | grep -i priority

Length of output: 834

---

🏁 Script executed:

#!/bin/bash
# Create priority:medium label with an orange color
gh label create "priority:medium" --description "Medium priority task" --color "ff9800"

# Now add the label to issue #2
gh issue edit 2 --add-label "priority:medium"

# Verify the issue was updated correctly
gh issue view 2 --json title,labels,url | jq '.labels[].name'

Length of output: 156

---

@unclesp1d3r I've successfully enhanced issue #2 with the following improvements:

Title Enhancement

New title: "Enhance ELF Dynamic Symbol Extraction with Library Mapping and Comprehensive Symbol Classification"

This is more descriptive and clearly communicates the scope beyond just basic extraction.

Body Enhancements

✅ Added detailed context explaining the current limitations in src/container/elf.rs:

• No library mapping from DT_NEEDED entries
• Limited to function symbols only (missing data objects, TLS variables, etc.)
• Incomplete dynamic section parsing
• Missing symbol visibility handling

✅ Added technical background covering:

• ELF dynamic section structure and key entries (DT_NEEDED, DT_SYMTAB, etc.)
• Symbol classification rules (imports vs exports)
• Symbol types (functions, objects, TLS, IFuncs)

✅ Added comprehensive proposed solution with:

• Step-by-step implementation plan
• Code structure examples showing new helper methods
• Specific symbol types to handle
• Data structure extensions if needed

✅ Enhanced acceptance criteria with specific, measurable goals including:

• DT_NEEDED parsing
• Library-to-symbol mapping
• All symbol type extraction
• Proper handling of weak symbols and visibility
• Comprehensive testing with insta snapshots

Metadata Updates

✅ Created and applied priority:medium label (orange color)
✅ Retained existing labels:

• area:analyzer
• type:enhancement
• status:backlog
• lang:rust
• needs:tests

The issue is now ready for implementation with clear technical direction and acceptance criteria! You can view it at: #2