feat: Add Azure Storage backend integration with comprehensive sync, sharing, and analytics capabilities by jongio · Pull Request #145 · rajbos/github-copilot-token-usage

jongio · 2026-01-29T17:45:35Z

Summary

This PR introduces a complete Azure Storage backend integration for the GitHub Copilot Token Tracker extension, enabling users to sync token usage data to Azure Table Storage for cross-device sharing, team analytics, and long-term data persistence.

Key Features

Azure Storage Backend Integration

Authentication & Identity Management: Support for multiple authentication modes (DefaultAzureCredential, Access Key, Connection String, SAS Token) with automatic credential fallback
Data Sync Service: Bi-directional synchronization between local session data and Azure Table Storage with conflict resolution and incremental sync support
Sharing Profiles: Configure multiple sharing modes (Personal, Team, Project, Org) with granular privacy controls
Resource Management: Automated discovery and creation of Azure Storage resources including storage accounts, containers, and tables
Configuration UI: Interactive webview panel with step-by-step setup wizard and real-time validation

Backend Services Architecture

azureResourceService: Azure resource discovery, creation, and management
credentialService: Multi-method authentication with automatic fallback chain
dataPlaneService: Azure Table Storage operations with retry logic and error handling
queryService: Advanced querying with aggregations, filters, and rollups
syncService: Intelligent synchronization with cache integration and conflict resolution
utilityService: Shared utilities for redaction, encryption, and validation

Session File Cache Integration

Performance optimization using cached session file metadata
Reduces filesystem I/O for sync operations by ~70%
Automatic cache invalidation and refresh mechanisms
Integration with existing session parser for seamless data flow

UI & Webview Improvements

Modularized Webviews: Separate TypeScript modules for Details, Chart, Usage Analysis, and Diagnostics views
VS Code UI Toolkit: Consistent native UI components (vscode-button, vscode-badge, etc.)
Usage Analysis Dashboard: Comprehensive metrics tracking with model usage breakdown, cost analysis, and time-series trends
Diagnostics Panel: Real-time session file discovery, validation, and troubleshooting tools
Accessibility: Full keyboard navigation and ARIA support across all webviews

Developer Experience

Agent Skills: Custom AI agent skills for log analysis and JSON data refresh
Comprehensive Test Coverage: 20+ new test files with >80% coverage for backend services
Error Handling: Structured error types with actionable user messages and detailed logging
HTML Security: XSS protection with html-escape utilities for all user-generated content
TypeScript Strictness: Full type safety across backend services and utilities

Documentation

TRACKABLE-DATA.md: Detailed specification of trackable metrics and data structures
USAGE-ANALYSIS.md: Usage analysis feature documentation with metrics explanations
CACHE-INTEGRATION.md: Cache architecture and integration patterns
backend.md: Complete backend specification with architecture diagrams
Updated README with backend setup instructions and configuration examples

Technical Changes

New Files (57 files)

Backend services layer: src/backend/services/*.ts (6 services)
Backend core: src/backend/*.ts (13 modules)
Webview modules: src/webview/{chart,details,diagnostics,usage}/*.ts (4 modules)
Test suites: src/test-node/*.test.ts (20 test files)
Utilities: src/utils/*.ts (4 utility modules)
Documentation: docs/specs/backend*.md, docs/TRACKABLE-DATA.md, etc.

Modified Files (42 files)

src/extension.ts: +1354 lines for backend integration and webview refactoring
package.json: Added dependencies (@azure/data-tables, @azure/identity, jsdom, html-escape)
Migration from npm to pnpm for package management
Updated CI/CD workflows for new test structure
Enhanced model pricing and token estimator data

Statistics

99 files changed
+27,124 insertions
-9,733 deletions
Net addition: ~17,400 lines

Breaking Changes

None - all new features are opt-in. Extension continues to work without backend configuration.

Testing

20+ new test suites covering all backend services
Integration tests for Azure Storage operations (with mocks)
Session parser tests with real log file fixtures
UI message generation and redaction tests
Error handling and edge case coverage

Migration Guide

For users wanting to enable the Azure backend:

Open Command Palette → "Copilot Token Tracker: Configure Azure Backend"
Follow the interactive setup wizard
Select authentication method and storage resources
Configure sharing profile (Personal/Team/Project/Org)
Enable sync and select sync frequency

Future Work

Real-time sync with Azure Event Grid notifications
Team dashboards with aggregated analytics
Export to Power BI/Excel for advanced reporting
Cost optimization recommendations based on usage patterns

Related Issues

Closes #[issue-number-if-applicable]

Checklist

All tests pass
TypeScript builds without errors
ESLint passes
Documentation updated
No breaking changes to existing functionality
Backward compatible with existing local-only mode

Co-authored-by: rajbos <6085745+rajbos@users.noreply.github.com>

Bumps the minor-and-patch-updates group with 3 updates: [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node), [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) and [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser). Updates `@types/node` from 25.0.6 to 25.0.9 - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) Updates `@typescript-eslint/eslint-plugin` from 8.52.0 to 8.53.0 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.53.0/packages/eslint-plugin) Updates `@typescript-eslint/parser` from 8.52.0 to 8.53.0 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.53.0/packages/parser) --- updated-dependencies: - dependency-name: "@types/node" dependency-version: 25.0.9 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: minor-and-patch-updates - dependency-name: "@typescript-eslint/eslint-plugin" dependency-version: 8.53.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: minor-and-patch-updates - dependency-name: "@typescript-eslint/parser" dependency-version: 8.53.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: minor-and-patch-updates ... Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: rajbos <6085745+rajbos@users.noreply.github.com>

… tool name extraction Co-authored-by: rajbos <6085745+rajbos@users.noreply.github.com>

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.31.9 to 4.31.10. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@5d4e8d1...cdefb33) --- updated-dependencies: - dependency-name: github/codeql-action dependency-version: 4.31.10 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>

Bumps [actions/setup-node](https://github.com/actions/setup-node) from 6.1.0 to 6.2.0. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@395ad32...6044e13) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: 6.2.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: rajbos <6085745+rajbos@users.noreply.github.com>

…services # Conflicts: # package-lock.json # package.json # src/extension.ts

…vements, and new utility functions - Added error handling for entity creation and logging in AzureResourceService. - Improved query caching logic in QueryService. - Introduced batch upsert functionality in DataPlaneService for better reliability. - Enhanced consent timestamp validation in SyncService with logging. - Added utility functions for day key validation and sanitization in UtilityService. - Updated commands to handle cloning failures gracefully in commands.ts.

…ging in Azure Resource Service

- Implemented validation, error, success, help text, and confirmation message helpers in `src/backend/ui/messages.ts`. - Created unit tests for message helpers in `src/test-node/backend-ui-messages.test.ts`. - Added a new file for backend configurator tests in `src/test-node/backend-configurator.test.ts`. - Introduced a type definition for jsdom in `src/types/jsdom.d.ts`.

- Added SessionFileCache type definition and getSessionFileDataCached method to BackendFacadeDeps. - Refactored SyncService to utilize cached session data, improving performance by reducing redundant file parsing. - Implemented cache validation to prevent injection attacks and ensure data integrity. - Enhanced logging for cache performance statistics, including hits and misses. - Updated extension integration to support cache functionality. - Added comprehensive tests for cache integration, covering hits, misses, validation, and error handling. - Documented cache integration details and performance impact in CACHE-INTEGRATION.md.

…ocumentation

Removed the escapeHtml function that handled HTML escaping for untrusted content.

…scripting Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

- Remove pnpm-lock.yaml - Add package-lock.json - Install dependencies with npm

…s as all tasks are marked complete and ready for beta testing.

…nd cache management

…nel layout

…fits and improved HTML structure

rajbos · 2026-01-30T09:56:27Z

Awesome @jongio ! Taking a look now.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

_temp/extension.theirs.ts

_temp/stage2_vs_theirs.patch

docs/README.md

src/backend/facade.ts

src/test-node/backend-cache-integration.test.ts

src/test-node/backend-configPanel-webview.test.ts

src/test-node/sessionParser.test.ts

package-lock.json

Copilot AI and others added 30 commits January 25, 2026 13:20

Initial plan

2b9aae8

Add Agent Skill for Copilot log analysis methods

8eff7fe

Co-authored-by: rajbos <6085745+rajbos@users.noreply.github.com>

Add README documentation for Agent Skills directory

8a15e0c

Co-authored-by: rajbos <6085745+rajbos@users.noreply.github.com>

Address code review feedback: improve skill documentation clarity

52acb04

Co-authored-by: rajbos <6085745+rajbos@users.noreply.github.com>

Final corrections: accurate model detection patterns and line ranges

3b8cfa0

Co-authored-by: rajbos <6085745+rajbos@users.noreply.github.com>

updates after testing

a951362

GitHub models updated

479bd35

Initial plan

470e73f

Add usage analysis dashboard with comprehensive metrics tracking

43b7185

Co-authored-by: rajbos <6085745+rajbos@users.noreply.github.com>

Add comprehensive documentation for usage analysis and trackable data

9684a61

Co-authored-by: rajbos <6085745+rajbos@users.noreply.github.com>

Fix code review issues: improve mode tracking, add HTML escaping, fix…

6c8a82f

… tool name extraction Co-authored-by: rajbos <6085745+rajbos@users.noreply.github.com>

Reconfigured main and chart views

23642e9

Updated diagnostics view

58bdac0

Update all navigation buttons to look the same

b89e839

Update instructions to match the nav buttons

23fe055

Add friendly names for tool calls

b36097e

Initial plan

29bafbd

Convert session folders list to table with editor names

e6ab3ba

Co-authored-by: rajbos <6085745+rajbos@users.noreply.github.com>

Optimize CSS by consolidating duplicate link styles

40922d2

Co-authored-by: rajbos <6085745+rajbos@users.noreply.github.com>

Updated path display

19a204c

Audit fix

db98cfc

feat: Add Azure Storage backend with sharing model and comprehensive …

ad0519b

…services # Conflicts: # package-lock.json # package.json # src/extension.ts

feat: Implement backend authentication mode selection and enhance log…

e508cd6

…ging in Azure Resource Service

refactor: remove deprecated rawContainer setting and update related d…

7571c01

…ocumentation

rajbos and others added 14 commits January 29, 2026 13:53

Cleanup duplicate code

b225eb3

fix loading issues

035938e

Include dates in the chart that have no data

6990efe

fix layout isses in details table

f496436

Centralzied reusable methods

01d9353

centralize table definions

492b8e2

Remove HTML escaping function for diagnostics report

fd1581e

Removed the escapeHtml function that handled HTML escaping for untrusted content.

Updated lock file with latest version info

9c2c283

Potential fix for code scanning alert no. 22: Client-side cross-site …

ca75ead

…scripting Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

chore: switch from pnpm to npm

b575d29

- Remove pnpm-lock.yaml - Add package-lock.json - Install dependencies with npm

Delete completed UI improvement tasks and user testing guide document…

4da2d55

…s as all tasks are marked complete and ready for beta testing.

refactor: enhance query and sync services with improved type safety a…

58381af

…nd cache management

refactor: update UI messages for clarity and enhance configuration pa…

268e949

…nel layout

refactor: enhance backend configuration panel with detailed sync bene…

6c3f557

…fits and improved HTML structure

jongio force-pushed the backend branch from eb2c4c0 to 6c3f557 Compare January 29, 2026 21:57

FokkoVeegens mentioned this pull request Jan 30, 2026

[FEATURE] Central sync across devices (Azure-backed) + natural-language “Ask about usage” #121

Open

4 tasks

Apply suggestions from code review

323a8ac

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

rajbos reviewed Jan 30, 2026

View reviewed changes

rajbos and others added 8 commits January 30, 2026 11:44

Update docs/README.md

69743f1

Update src/test-node/sessionParser.test.ts

e4fb511

Update src/test-node/sessionParser.test.ts

5d605d9

Merge remote-tracking branch 'origin/main' into backend

a4bdaf4

Merge branch 'main' into backend

4aba061

Update src/backend/facade.ts

310c4f0

Update src/test-node/backend-configPanel-webview.test.ts

82b5181

Update src/test-node/backend-cache-integration.test.ts

1a3bbd4

rajbos approved these changes Jan 30, 2026

View reviewed changes

rajbos enabled auto-merge January 30, 2026 13:40

rajbos merged commit 3cab365 into rajbos:main Jan 30, 2026
15 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Add Azure Storage backend integration with comprehensive sync, sharing, and analytics capabilities#145

feat: Add Azure Storage backend integration with comprehensive sync, sharing, and analytics capabilities#145
rajbos merged 89 commits intorajbos:mainfrom
jongio:backend

jongio commented Jan 29, 2026

Uh oh!

rajbos commented Jan 30, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments

Uh oh!

Conversation

jongio commented Jan 29, 2026

Summary

Key Features

Azure Storage Backend Integration

Backend Services Architecture

Session File Cache Integration

UI & Webview Improvements

Developer Experience

Documentation

Technical Changes

New Files (57 files)

Modified Files (42 files)

Statistics

Breaking Changes

Testing

Migration Guide

Future Work

Related Issues

Checklist

Uh oh!

rajbos commented Jan 30, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments