fix: support newer Gemini CLI and Copilot session formats

[jimyag]

Summary

• support Gemini CLI .jsonl session logs and nested chats/ subdirectories
• ignore Gemini warning messages and $set metadata updates while keeping the latest message version per id
• tolerate empty newer GitHub Copilot chat session files and add regression coverage for both parsers

Root Cause

Recent Gemini CLI releases changed chat persistence from a single .json object to incremental .jsonl logs and introduced nested subagent session paths. Newer Copilot session files can also omit fields that the parser previously required even when the session has no requests.

Validation

• cargo fmt --all --quiet
• cargo build --quiet
• cargo test --quiet
• cargo clippy --quiet -- -D warnings
• cargo doc --quiet

Fixes #154

Summary by CodeRabbit

• New Features

  • Support for parsing Gemini CLI JSONL chat logs with per-message-id deduplication (latest wins)
  • Recognition and exclusion of Warning message type from conversation streams

• Bug Fixes

  • Copilot session parser no longer fails when optional metadata fields are absent

• Tests

  • Added tests covering Gemini JSONL handling, warning-message behavior, and Copilot parsing of empty-request sessions

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: da49d7df-0ebe-4f5e-8855-e900025f733f

📥 Commits

Reviewing files that changed from the base of the PR and between d2a7dab and 27639d8.

📒 Files selected for processing (1)

• src/analyzers/tests/copilot.rs

🚧 Files skipped from review as they are similar to previous changes (1)

• src/analyzers/tests/copilot.rs

---

📝 Walkthrough

Walkthrough

Adds JSONL support and per-message-ID deduplication to the Gemini CLI analyzer, expands chat-file discovery to include .json and .jsonl under chats/, recognizes a Warning message type, and makes several Copilot session fields optional to avoid deserialization failures. Tests added for both changes.

Changes

Cohort / File(s)	Summary
Copilot Optional Fields src/analyzers/copilot.rs	Made requester_username, responder_username, and initial_location optional (Option<String>) with #[serde(default)] so missing fields deserialize as None.
Copilot Tests src/analyzers/tests/copilot.rs	Added unit test verifying parsing of a Copilot session JSON with empty requests and absent optional metadata yields zero parsed messages.
Gemini CLI Analyzer src/analyzers/gemini_cli.rs	Added JSONL parsing path with per-id latest-message deduplication while preserving ordering; dispatches parse behavior by file extension (.jsonl vs .json); refactored session→message conversion into messages_from_session helper and added Warning message type recognition; updated chat-file discovery to accept .json and .jsonl under chats/ (including nested subdirs).
Gemini CLI Tests src/analyzers/tests/gemini_cli.rs	Added async tests: one asserting type: "warning" entries are ignored (only user/assistant preserved); another asserting JSONL handling chooses the latest $set-updated message by id and preserves token/reasoning/tool stats and that generated session path is considered valid.

Sequence Diagram(s)

sequenceDiagram
    participant Reader as File Reader
    participant Parser as JSON/JSONL Parser
    participant Deduper as ID Deduplicator
    participant Converter as messages_from_session
    participant Storage as Message Sink

    Reader->>Parser: Read file (detect .json / .jsonl)
    alt .jsonl
        Parser->>Parser: Stream parse lines -> events
        Parser->>Deduper: Emit message events (id, payload)
        Deduper->>Deduper: Keep latest per id (by timestamp/$set)
        Deduper->>Converter: Emit ordered latest messages
    else .json
        Parser->>Converter: Parse full session object
    end
    Converter->>Storage: Convert to ConversationMessage list (compute conversation_hash)

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

🐰 I nibbled logs in JSONL light,

deduped the hops that came at night.
Warnings whisper, fields set free—
Copilot smiles, parsed gracefully.
Hooray, the chats dance back to me!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 52.94% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: support newer Gemini CLI and Copilot session formats' accurately and concisely summarizes the main change: adding support for newer versions of both tools' session file formats.
Linked Issues check	✅ Passed	All coding requirements from issue #154 are met: JSONL parsing with deduplication by id, nested chats/ directory support, warning message filtering, $set metadata handling, and Copilot optional field tolerance with regression tests.
Out of Scope Changes check	✅ Passed	All changes directly support the linked issue objectives: JSONL parsing, nested directory discovery, message deduplication, warning filtering, optional field tolerance, and comprehensive test coverage—no out-of-scope modifications detected.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Review rate limit: 0/1 reviews remaining, refill in 60 minutes.}

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[jimyag]

before

after

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/analyzers/gemini_cli.rs (1)

416-422: ⚠️ Potential issue | 🟠 Major

Update get_data_glob_patterns to include JSONL files and nested chat paths.

Line 421 only emits .../chats/*.json, omitting .jsonl files and nested sessions under chats/** directories. While discover_data_sources uses WalkDir with is_gemini_cli_chat_path validation (which correctly handles both formats and nested paths), the glob patterns should match the actual data locations supported by the analyzer. Other similar analyzers (copilot_cli, codex_cli, claude_code) all include nested paths with ** or explicit */* patterns and multiple file formats.

Suggested patch

 fn get_data_glob_patterns(&self) -> Vec<String> {
     let mut patterns = Vec::new();

     if let Some(home_dir) = dirs::home_dir() {
         let home_str = home_dir.to_string_lossy();
+        patterns.push(format!("{home_str}/.gemini/tmp/*/chats/*.json"));
+        patterns.push(format!("{home_str}/.gemini/tmp/*/chats/*.jsonl"));
+        patterns.push(format!("{home_str}/.gemini/tmp/*/chats/**/*.json"));
+        patterns.push(format!("{home_str}/.gemini/tmp/*/chats/**/*.jsonl"));
     }

     patterns
 }

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/analyzers/gemini_cli.rs` around lines 416 - 422, The current
get_data_glob_patterns function only returns patterns matching chats/*.json;
update it to also match .jsonl files and nested chat directories by adding glob
entries such as "{home_str}/.gemini/tmp/**/chats/**/*.json",
"{home_str}/.gemini/tmp/**/chats/**/*.jsonl" (and mirror any non-** variants you
use elsewhere like "{home_str}/.gemini/tmp/*/chats/**/*.json" if you prefer
single-level wildcards) so the returned Vec<String> covers both .json and .jsonl
and nested sessions; modify the patterns pushed inside get_data_glob_patterns
accordingly.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/analyzers/tests/copilot.rs`:
- Around line 203-232: Update the test
test_parse_empty_copilot_session_without_requester_username to exercise the
parser tolerance by removing all formerly-required fields instead of only
responderUsername: construct the JSON fixture passed to
parse_copilot_session_file with responderUsername, initialLocation, and the
other previously-required field (e.g., inputState) omitted (or add separate
tests that omit each required field combination) so the test asserts the parser
accepts sessions missing all those optional fields.

---

Outside diff comments:
In `@src/analyzers/gemini_cli.rs`:
- Around line 416-422: The current get_data_glob_patterns function only returns
patterns matching chats/*.json; update it to also match .jsonl files and nested
chat directories by adding glob entries such as
"{home_str}/.gemini/tmp/**/chats/**/*.json",
"{home_str}/.gemini/tmp/**/chats/**/*.jsonl" (and mirror any non-** variants you
use elsewhere like "{home_str}/.gemini/tmp/*/chats/**/*.json" if you prefer
single-level wildcards) so the returned Vec<String> covers both .json and .jsonl
and nested sessions; modify the patterns pushed inside get_data_glob_patterns
accordingly.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 672706b6-d339-4247-8d2a-16cb81c728d8

📥 Commits

Reviewing files that changed from the base of the PR and between 9db43c2 and d2a7dab.

📒 Files selected for processing (4)

• src/analyzers/copilot.rs
• src/analyzers/gemini_cli.rs
• src/analyzers/tests/copilot.rs
• src/analyzers/tests/gemini_cli.rs

[jimyag]

@coderabbitai review it

[coderabbitai]

@jimyag Sure, I'll review the changes right away!

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[mike1858]

@coderabbitai Review

[coderabbitai]

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[mike1858]

Great, thank you!