feat(core): Go scanner with tree-sitter infrastructure

[prosdev]

Summary

Implement core Go language scanner using tree-sitter WASM, establishing reusable infrastructure for future language support.

Background

Part of Epic #128: Go Language Support

Tasks

Tree-sitter Infrastructure

• Add web-tree-sitter dependency
• Add tree-sitter-go WASM grammar
• Create tree-sitter utility module for query execution
• Handle grammar loading and caching

Go Scanner Core

• Implement GoScanner class implementing Scanner interface
• Extract top-level functions with signatures
• Extract struct definitions
• Extract interface definitions
• Extract type aliases
• Extract exported constants and variables
• Detect exported status (capital letter convention)

Basic Testing

• Unit tests for each extraction type
• Integration test with sample Go files
• Test error handling for malformed Go files

Technical Notes

Dependencies:

{
  "web-tree-sitter": "^0.22.x",
  "tree-sitter-go": "^0.21.x"
}

File Extensions: .go

Scanner Interface:

class GoScanner implements Scanner {
  readonly language = "go";
  readonly capabilities: ScannerCapabilities = {
    syntax: true,
    types: true,
    documentation: true,
  };
  
  canHandle(filePath: string): boolean;
  async scan(files: string[], repoRoot: string): Promise<Document[]>;
}

Acceptance Criteria

• GoScanner extracts functions, structs, interfaces, types from .go files
• Tree-sitter infrastructure is modular and reusable
• Basic test coverage for core extraction
• Scanner can be instantiated and used standalone

References

• Scanner types: packages/core/src/scanner/types.ts
• TypeScript scanner example: packages/core/src/scanner/typescript.ts

[prosdev]

Design Decision: Tree-sitter Query Approach

Research Summary

Investigated how other AI coding tools handle Go language support:

Aider (most relevant - widely used AI coding tool):

• Uses tree-sitter via tree-sitter-language-pack
• Extracts symbols using tags.scm query files rather than manual AST traversal
• Query-based approach is declarative and maintainable

Other tools:

• Sourcegraph, Zed Editor: Also use tree-sitter with queries
• This is the established pattern for multi-language support

Decision

Use tree-sitter queries instead of manual AST traversal

This differs from our TypeScript scanner (which uses ts-morph AST traversal) but aligns with:

1. Industry standard for multi-language support
2. More maintainable - declarative patterns vs imperative code
3. Reusable infrastructure - same query approach works for Python/Rust later
4. Well-documented query patterns exist in tree-sitter-go/queries/tags.scm

Implementation Plan

Dependencies:
- web-tree-sitter (WASM-based, no native bindings needed)
- tree-sitter-go WASM grammar (from tree-sitter-wasms or built)

Approach:
1. Create tree-sitter utility module for query execution
2. Port/adapt tags.scm patterns for our Document extraction needs
3. GoScanner uses queries to extract: functions, methods, structs, interfaces, types

Example Query Pattern (from tree-sitter-go)

; Functions
(function_declaration
  name: (identifier) @name) @definition.function

; Methods  
(method_declaration
  name: (field_identifier) @name
  receiver: (parameter_list
    (parameter_declaration
      type: [(pointer_type (type_identifier) @receiver)
             (type_identifier) @receiver]))) @definition.method

; Structs
(type_declaration
  (type_spec
    name: (type_identifier) @name
    type: (struct_type))) @definition.class

; Interfaces
(type_declaration
  (type_spec
    name: (type_identifier) @name
    type: (interface_type))) @definition.interface

Trade-offs

Aspect	Query Approach	Manual AST (like TS scanner)
Maintainability	✅ Declarative patterns	❌ Imperative traversal
Reusability	✅ Same pattern for all languages	❌ Language-specific code
Learning curve	⚠️ S-expression syntax	✅ Familiar TS code
Flexibility	⚠️ Limited to query capabilities	✅ Full control
Performance	✅ Optimized by tree-sitter	⚠️ Depends on implementation

Conclusion: Query approach is the right choice for multi-language support infrastructure.