Enhance ELF symbol extraction with comprehensive type support and visibility filtering

[Copilot]

The ELF parser extracted only STT_FUNC symbols and lacked library dependency parsing and visibility filtering, limiting import/export analysis compared to the PE parser.

Changes

Symbol Type Support

• Extended import/export extraction to handle STT_TLS (thread-local storage) and STT_GNU_IFUNC (indirect functions) in addition to existing STT_FUNC, STT_OBJECT, and STT_NOTYPE

Visibility Filtering

• Exports now filter symbols marked STV_HIDDEN or STV_INTERNAL, exposing only externally visible symbols

Library Dependency Extraction

• Added extract_needed_libraries() to parse DT_NEEDED entries from the dynamic section
• Returns required shared library names (e.g., libc.so.6)
• Foundation for future symbol-to-library mapping via version symbols or PLT/GOT analysis

Example

// Before: only STT_FUNC symbols extracted
if sym.st_type() == goblin::elf::sym::STT_FUNC { ... }

// After: comprehensive symbol type coverage
if sym.st_type() == goblin::elf::sym::STT_FUNC
    || sym.st_type() == goblin::elf::sym::STT_OBJECT
    || sym.st_type() == goblin::elf::sym::STT_TLS
    || sym.st_type() == goblin::elf::sym::STT_GNU_IFUNC
    || sym.st_type() == goblin::elf::sym::STT_NOTYPE
{
    // Visibility check for exports
    if sym.st_visibility() != goblin::elf::sym::STV_HIDDEN
        && sym.st_visibility() != goblin::elf::sym::STV_INTERNAL
    { ... }
}

Testing

• Unit tests for symbol type and visibility constants
• Integration test validating library extraction with real binaries
• Snapshot test capturing symbol extraction output for regression detection

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Enhance ELF Dynamic Symbol Extraction with Library Mapping and Comprehensive Symbol Classification</issue_title>
<issue_description>## Summary

Enhance the existing ELF import/export extraction to comprehensively parse the dynamic section, extract library dependencies (DT_NEEDED entries), map symbols to their originating libraries, and improve symbol classification beyond just functions.

Context

The current ELF parser in src/container/elf.rs provides basic import/export extraction by analyzing the dynamic symbol table (.dynsym). However, it has several limitations:

1. No library mapping: Imports don't identify which shared library they come from (DT_NEEDED entries are not parsed)
2. Limited symbol types: Only extracts function symbols (STT_FUNC), missing data objects, TLS variables, etc.
3. Incomplete dynamic section parsing: The dynamic section contains additional metadata (version requirements, symbol versioning, etc.) that isn't currently extracted
4. Missing symbol visibility: Doesn't account for symbol visibility (STV_DEFAULT, STV_HIDDEN, etc.)

This enhancement extends the existing functionality to provide more complete import/export analysis, matching the comprehensiveness of the PE parser implementation.

Technical Background

ELF Dynamic Section Structure:

• Contains DT_NEEDED entries that specify required shared libraries
• Includes DT_SYMTAB, DT_STRTAB, and DT_HASH/DT_GNU_HASH for symbol resolution
• May contain version information via DT_VERNEED, DT_VERDEF, and DT_VERSYM

Symbol Classification:

• Imports: Undefined symbols (SHN_UNDEF) that need to be resolved at link/load time
• Exports: Defined symbols with global/weak binding available for other modules
• Symbol types: Functions (STT_FUNC), objects (STT_OBJECT), TLS (STT_TLS), IFuncs (STT_GNU_IFUNC)

Proposed Solution

Implementation Steps

1. Parse DT_NEEDED entries from dynamic section

   • Extract all required library names
   • Build a mapping of libraries for symbol attribution

2. Enhance import extraction (extract_imports)

   • Keep current undefined symbol detection
   • Extend to handle all symbol types (not just STT_FUNC):
     • STT_OBJECT (data objects)
     • STT_TLS (thread-local storage)
     • STT_GNU_IFUNC (indirect functions)
   • Attempt to map symbols to libraries using version information when available
   • Preserve symbol visibility and binding information

3. Enhance export extraction (extract_exports)

   • Include all globally visible defined symbols
   • Add support for weak symbols (STB_WEAK)
   • Include symbol type information in ExportInfo
   • Filter out hidden symbols (STV_HIDDEN, STV_INTERNAL)

4. Extend data structures if needed

   • May need to extend ImportInfo to include symbol type and version
   • May need to extend ExportInfo to include symbol type and binding

5. Add comprehensive unit tests

   • Test DT_NEEDED extraction with mock ELF data
   • Test symbol classification for various types
   • Test edge cases (weak symbols, versioned symbols, hidden symbols)
   • Use insta snapshots for symbol extraction results

6. Add integration tests with real binaries

   • Test with sample ELF binaries (e.g., /bin/ls equivalent)
   • Verify correct library mapping
   • Validate symbol counts and types

Code Structure

// New helper method
fn extract_needed_libraries(&self, elf: &Elf) -> Vec<String> {
    // Parse DT_NEEDED entries from elf.dynamic
}

// Enhanced import extraction
fn extract_imports(&self, elf: &Elf, libraries: &[String]) -> Vec<ImportInfo> {
    // Current logic + handle all symbol types + library mapping
}

// Enhanced export extraction  
fn extract_exports(&self, elf: &Elf) -> Vec<ExportInfo> {
    // Current logic + handle weak symbols + filter hidden
}

Requirements

4.2, 4.3

Acceptance Criteria

• ✅ Parse DT_NEEDED entries and extract library dependencies
• ✅ Map imported symbols to their originating libraries where possible
• ✅ Extract all relevant symbol types (functions, objects, TLS, IFuncs)
• ✅ Properly classify symbols as imports vs exports based on definition and binding
• ✅ Handle weak symbols and symbol visibility correctly
• ✅ Add comprehensive unit tests with cargo test passing
• ✅ Add integration tests with real ELF binaries
• ✅ Use insta for snapshot testing of symbol extraction
• ✅ Ensure cargo clippy -- -D warnings passes
• ✅ Update documentation in docs/ explaining the enhanced extraction
• ✅ Add benchmarks with criterion if performance-sensitive paths are added
• ✅ Update justfile recipes if new test targets are added
• ✅ Ensure CI passes

Dependencies

• Blocked by: Implement Intelligent ELF Section Classification for Targeted String Extraction #1 ELF Section Classification (appears to be already implemented based on codebase)

Task-ID

stringy-analyzer/elf-import-export-extraction</issue_desc...

• Fixes Enhance ELF Dynamic Symbol Extraction with Library Mapping and Comprehensive Symbol Classification #2

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[coderabbitai]

Caution

Review failed

The pull request is closed.

Summary by CodeRabbit

• New Features

  • Enhanced binary parsing with improved import/export extraction and library dependency analysis
  • Added performance benchmarking infrastructure for ELF parsing

• Tests

  • Added comprehensive integration tests with pre-compiled binary fixtures for ELF, Mach-O, and PE formats

• Chores

  • Updated to Rust 2024 edition with improved tooling configuration
  • Updated core dependencies and development tools for better stability and performance

Walkthrough

Adds extensive developer documentation and Rust project standards; updates CI/docs workflow and Cargo configuration; introduces fixture-based integration tests and Criterion ELF benchmarks; and significantly enhances ELF parsing with DT_NEEDED extraction, symbol versioning, symbol-to-library mapping, broader symbol classification, and related APIs/tests.

Changes

Cohort / File(s)	Summary
Developer Tooling — AI Command Workflows .cursor/commands/ci_check.md, .cursor/commands/code_rabbit.md, .cursor/commands/code_review.md, .cursor/commands/performance_tuning.md, .cursor/commands/security_hardening.md, .cursor/commands/update_llmstxt.md, .cursor/commands/work_next_task.md	Seven new procedural docs describing CI checks, CodeRabbit workflow, code review rules, performance tuning, security hardening, llms.txt updates, and task execution checklists.
Rust Coding Standards & Rules .cursor/rules/rust/cargo-toml.mdc, .cursor/rules/rust/configuration-management.mdc, .cursor/rules/rust/error-handling-patterns.mdc, .cursor/rules/rust/error-handling.mdc, .cursor/rules/rust/linting-rules.mdc, .cursor/rules/rust/performance-optimization.mdc, .cursor/rules/rust/rust-standards.mdc	Seven new rule/standards documents covering Cargo.toml conventions, CLI config validation, error-handling patterns, linting rules, performance optimization guidance, and overall Rust project standards.
Test Fixtures & Fixture Docs tests/fixtures/README.md, tests/fixtures/test_binary.c	Adds test fixtures README and C source used to generate ELF/Mach-O/PE test binaries (exported functions and libc imports).
Integration Tests — Binary Parsers tests/integration_elf.rs, tests/integration_macho.rs, tests/integration_pe.rs	New fixture-based integration tests validating import/export extraction, section classification, weights, and symbol-to-library behavior for ELF, Mach-O, and PE.
ELF Parser Implementation & API src/container/elf.rs, docs/src/binary-formats.md	Major ELF parser changes: extract_imports signature now takes libraries: &[String]; added extract_needed_libraries, get_symbol_providing_library, resolve_versym, parse_verneed_entry; added deduplication via seen_names; broadened symbol types (STT_FUNC, STT_OBJECT, STT_TLS, STT_GNU_IFUNC), visibility filtering, and updated docs describing new methods.
Benchmarks benches/elf.rs	Criterion benchmarks for ELF parsing (full parse, parse with imports, parse with exports).
Build / CI / Manifest .github/workflows/docs.yml, Cargo.toml, deny.toml	CI/docs workflow tweaks (mdbook version, rustdoc ordering), Cargo.toml updated to Rust 2024 and dependency bumps (clap, goblin, insta, tempfile) and bench target; deny.toml license and registry adjustments.
Tests Fixtures Index tests/fixtures/README.md	Documented fixture usage and rebuild instructions for test binaries across platforms.

Sequence Diagram(s)

sequenceDiagram
    participant Test as Integration Test
    participant Parser as ElfParser
    participant ELF as goblin::Elf
    participant Vers as VersionInfo

    Test->>Parser: call extract_imports(elf, libraries)
    Parser->>Parser: extract_needed_libraries(elf)
    Parser->>ELF: read DT_NEEDED, dynsym, versym, verneed
    ELF-->>Parser: libraries[], dynsyms[], versym_table, verneed_entries

    loop each undefined dynsym
        Parser->>Vers: resolve_versym(elf, sym_index)
        Vers-->>Parser: version_index or None
        Parser->>Vers: parse_verneed_entry(elf, version_index)
        Vers-->>Parser: (library_name, version) or None
        Parser->>Parser: get_symbol_providing_library(sym_index, libraries)
        Parser-->>Test: emit ImportInfo { name, library?, type, ... }
    end

    Test->>Parser: call extract_exports(elf)
    loop each defined symbol in syms/dynsyms
        alt visible and supported type
            Parser-->>Test: emit ExportInfo { name, type, visibility, ... }
        end
    end

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

• Areas needing careful review:
  • ELF versioning parsing and correctness in resolve_versym / parse_verneed_entry.
  • Symbol-to-library mapping logic in get_symbol_providing_library, including fallbacks and heuristics.
  • Symbol filtering across types/visibility and deduplication behavior (seen_names).
  • Integration tests and fixture paths/snapshots to ensure expectations match parser outputs.
  • Cargo/CI changes for compatibility with Rust 2024 and bumped deps.

Possibly related issues

• Establish Test Infrastructure with Multi-Format Binary Fixtures #34 — Adds test fixtures and format-specific integration tests; aligns with this PR’s switch to fixture-based testing and expanded integration coverage.

Possibly related PRs

• feat: Adds section-weighting and improves ELF imports #45 — Modifies src/container/elf.rs for import/export logic and library mapping; strong overlap with the same ELF extraction changes.
• fix: Resolve integration test failures and enhance static binary export extraction #51 — Also updates ELF symbol extraction and deduplication patterns; closely related at code level.
• feat: set up CI, linting, and documentation automation #44 — Updates CI/docs tooling and developer workflow files; related to the CI/doc changes in this PR.

Poem

🐰 In crowded bins where symbols hide, I hop and gently pry,

DT_NEEDED whispers names, and versym points the sky,
I map each import to its home, dedupe names with care,
Exports stand bright and visible—now every link’s aware. 🥕

Pre-merge checks and finishing touches

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately and specifically describes the main enhancement to ELF symbol extraction, covering the two primary changes: comprehensive symbol type support and visibility filtering.
Description check	✅ Passed	The description is well-structured, clearly explaining the problem, changes made (symbol types, visibility filtering, library extraction), and including concrete code examples demonstrating the improvements.
Linked Issues check	✅ Passed	The PR comprehensively implements all major requirements from issue #2: DT_NEEDED parsing, symbol type expansion (STT_TLS, STT_GNU_IFUNC), visibility filtering (STV_HIDDEN, STV_INTERNAL), library extraction methods, integration tests, and snapshot testing with insta.
Out of Scope Changes check	✅ Passed	Changes are well-scoped to the ELF symbol extraction enhancement. Workflow documentation and Rust project configuration files (.cursor/commands and .cursor/rules) appear to be foundational infrastructure setup and are not directly part of the functional enhancement but support the development process.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 6402946 and 321024b.

⛔ Files ignored due to path filters (2)

• tests/fixtures/test_binary_pe.exe is excluded by !**/*.exe
• tests/snapshots/integration_elf__elf_symbol_extraction.snap is excluded by !**/*.snap

📒 Files selected for processing (25)

• .cursor/commands/ci_check.md (1 hunks)
• .cursor/commands/code_rabbit.md (1 hunks)
• .cursor/commands/code_review.md (1 hunks)
• .cursor/commands/performance_tuning.md (1 hunks)
• .cursor/commands/security_hardening.md (1 hunks)
• .cursor/commands/update_llmstxt.md (1 hunks)
• .cursor/commands/work_next_task.md (1 hunks)
• .cursor/rules/rust/cargo-toml.mdc (1 hunks)
• .cursor/rules/rust/configuration-management.mdc (1 hunks)
• .cursor/rules/rust/error-handling-patterns.mdc (1 hunks)
• .cursor/rules/rust/error-handling.mdc (1 hunks)
• .cursor/rules/rust/linting-rules.mdc (1 hunks)
• .cursor/rules/rust/performance-optimization.mdc (1 hunks)
• .cursor/rules/rust/rust-standards.mdc (1 hunks)
• .github/workflows/docs.yml (2 hunks)
• Cargo.toml (2 hunks)
• benches/elf.rs (1 hunks)
• deny.toml (3 hunks)
• docs/src/binary-formats.md (2 hunks)
• src/container/elf.rs (6 hunks)
• tests/fixtures/README.md (1 hunks)
• tests/fixtures/test_binary.c (1 hunks)
• tests/integration_elf.rs (1 hunks)
• tests/integration_macho.rs (1 hunks)
• tests/integration_pe.rs (1 hunks)

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[dosubot]

Related Documentation

Checked 9 published document(s) in 1 knowledge base(s). No updates required.

^{How did I do? Any feedback?}  

[unclesp1d3r]

@coderabbitai full review

[coderabbitai]

✅ Actions performed

Full review triggered.

[coderabbitai]

✅ Actions performed

Full review triggered.